Jul 07 15:44:24.200904 ip-10-0-136-171 systemd[1]: Starting Kubernetes Kubelet... Jul 07 15:44:24.612058 ip-10-0-136-171 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:44:24.612058 ip-10-0-136-171 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 15:44:24.612058 ip-10-0-136-171 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:44:24.612058 ip-10-0-136-171 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 15:44:24.612058 ip-10-0-136-171 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:44:24.612951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.612859 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 15:44:24.616534 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616512 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:44:24.616534 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616529 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:44:24.616534 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616535 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:44:24.616534 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616539 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:44:24.616534 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616543 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616547 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616551 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616555 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616558 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616562 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616567 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616570 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616591 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616595 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616603 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616607 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616611 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616615 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616618 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616622 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616625 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616629 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616633 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616637 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:44:24.616849 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616640 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616644 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616648 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616652 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616656 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616659 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616664 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616668 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616672 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616676 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616680 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616687 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616693 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616698 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616705 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616709 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616714 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616718 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616723 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:44:24.617695 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616727 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616731 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616736 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616741 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616745 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616749 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616754 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616758 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616762 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616767 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616773 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616779 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616784 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616789 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616795 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616801 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616807 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616813 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616818 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:44:24.618532 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616831 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616836 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616841 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616845 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616849 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616853 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616856 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616860 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616865 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616871 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616875 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616879 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616883 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616887 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616891 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616895 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616899 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616903 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616907 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616912 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:44:24.619348 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616916 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616920 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616923 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616927 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616931 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616936 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616940 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616944 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616948 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616952 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616956 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616960 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616964 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616968 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616972 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616976 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.616980 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618156 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618166 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618171 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:44:24.620216 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618177 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618181 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618186 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618191 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618196 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618200 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618203 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618208 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618212 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618216 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618220 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618224 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618229 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618233 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618237 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618241 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618245 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618250 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618253 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:44:24.621097 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618258 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618262 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618266 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618270 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618274 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618279 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618284 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618288 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618293 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618297 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618301 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618305 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618309 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618314 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618318 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618322 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618327 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618331 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618335 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618339 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:44:24.621662 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618343 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618347 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618351 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618355 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618359 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618363 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618368 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618372 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618376 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618380 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618384 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618389 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618393 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618397 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618401 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618405 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618409 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618418 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618424 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:44:24.622278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618429 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618435 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618439 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618444 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618448 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618452 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618457 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618461 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618465 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618470 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618475 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618479 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618483 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618487 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618491 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618498 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618506 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618510 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618517 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:44:24.622967 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618522 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618526 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618530 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618534 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618538 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618542 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618546 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618550 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618554 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618558 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618562 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618566 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618599 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618605 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618620 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618624 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618628 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618632 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.618636 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619209 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:44:24.623464 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619219 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619225 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619231 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619236 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619241 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619245 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619250 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619255 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619260 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619264 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619270 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619275 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619280 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619284 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619288 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619292 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619296 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619300 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619304 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:44:24.624135 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619307 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619311 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619316 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619320 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619324 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619329 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619335 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619339 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619344 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619347 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619351 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619356 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619360 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619363 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619368 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619372 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619376 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619381 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619385 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619390 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:44:24.624692 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619394 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619398 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619402 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619408 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619413 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619417 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619421 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619425 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619430 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619434 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619438 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619442 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619446 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619450 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619454 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619458 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619462 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619467 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619472 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:44:24.625191 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619477 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619480 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619484 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619488 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619492 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619496 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619500 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619504 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619510 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619514 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619518 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619523 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619527 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619531 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619535 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619539 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619543 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619547 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619552 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619556 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:44:24.625682 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619560 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619565 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619570 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619593 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619597 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619601 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619605 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619609 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619613 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619617 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619621 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619627 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619631 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619635 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619639 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619643 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619647 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619650 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619655 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.619659 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619756 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 15:44:24.626159 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619772 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619782 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619789 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619796 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619801 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619807 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619819 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619825 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619829 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619835 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619840 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619844 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619852 2585 flags.go:64] FLAG: --cgroup-root="" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619857 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619862 2585 flags.go:64] FLAG: --client-ca-file="" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619866 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619870 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619877 2585 flags.go:64] FLAG: --cluster-domain="" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619881 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619886 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619891 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619896 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619904 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 15:44:24.626693 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619908 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619913 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619918 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619923 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619927 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619932 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619937 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619941 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619947 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619952 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619957 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619961 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619965 2585 flags.go:64] FLAG: --enable-server="true" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619971 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619979 2585 flags.go:64] FLAG: --event-burst="100" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619984 2585 flags.go:64] FLAG: --event-qps="50" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619988 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619993 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.619997 2585 flags.go:64] FLAG: --eviction-hard="" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620003 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620007 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620013 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620018 2585 flags.go:64] FLAG: --eviction-soft="" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620023 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620028 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 15:44:24.627233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620032 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620037 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620041 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620046 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620050 2585 flags.go:64] FLAG: --feature-gates="" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620056 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620063 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620068 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620073 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620078 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620082 2585 flags.go:64] FLAG: --help="false" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620087 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-136-171.ec2.internal" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620092 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620096 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620101 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620106 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620112 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620116 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620121 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620125 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620129 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620135 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620140 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620145 2585 flags.go:64] FLAG: --kube-reserved="" Jul 07 15:44:24.627959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620149 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620154 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620159 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620163 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620169 2585 flags.go:64] FLAG: --lock-file="" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620173 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620178 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620183 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620192 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620197 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620201 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620205 2585 flags.go:64] FLAG: --logging-format="text" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620210 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620215 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620221 2585 flags.go:64] FLAG: --manifest-url="" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620225 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620231 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620236 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620242 2585 flags.go:64] FLAG: --max-pods="110" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620247 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620251 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620256 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620260 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620265 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620270 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 15:44:24.628558 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620274 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620285 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620290 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620295 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620300 2585 flags.go:64] FLAG: --pod-cidr="" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620305 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620314 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620318 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620323 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620327 2585 flags.go:64] FLAG: --port="10250" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620332 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620336 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0567d843e1763ddd5" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620342 2585 flags.go:64] FLAG: --qos-reserved="" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620347 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620352 2585 flags.go:64] FLAG: --register-node="true" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620356 2585 flags.go:64] FLAG: --register-with-taints="" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620362 2585 flags.go:64] FLAG: --registry-burst="10" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620366 2585 flags.go:64] FLAG: --registry-qps="5" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620370 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620374 2585 flags.go:64] FLAG: --reserved-memory="" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620380 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620389 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620393 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620398 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620403 2585 flags.go:64] FLAG: --runonce="false" Jul 07 15:44:24.629382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620407 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620412 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620416 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620420 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620425 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620430 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620434 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620439 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620443 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620448 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620452 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620456 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620461 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620467 2585 flags.go:64] FLAG: --system-cgroups="" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620472 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620481 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620485 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620489 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620495 2585 flags.go:64] FLAG: --tls-min-version="" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620500 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620505 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620509 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620514 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620518 2585 flags.go:64] FLAG: --v="2" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620524 2585 flags.go:64] FLAG: --version="false" Jul 07 15:44:24.630156 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620530 2585 flags.go:64] FLAG: --vmodule="" Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620535 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.620541 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620739 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620749 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620755 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620761 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620765 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620770 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620775 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620802 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620809 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620814 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620819 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620823 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620839 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620844 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620849 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620854 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:44:24.630797 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620859 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620863 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620870 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620876 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620881 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620885 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620890 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620894 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620899 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620904 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620908 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620912 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620916 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620921 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620925 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620929 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620935 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620939 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620944 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:44:24.631257 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620948 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620952 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620956 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620960 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620965 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620969 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620972 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620976 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620980 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620985 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620989 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620993 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.620997 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621001 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621007 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621012 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621017 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621022 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621026 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621030 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:44:24.631731 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621034 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621038 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621042 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621046 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621050 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621054 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621058 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621062 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621066 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621072 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621076 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621080 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621084 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621088 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621092 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621096 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621100 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621103 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621107 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621111 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:44:24.632247 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621115 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621119 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621123 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621127 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621133 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621138 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621142 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621146 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621150 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621155 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621159 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621163 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621167 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621172 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621176 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621180 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621184 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621188 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621192 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621196 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:44:24.632768 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621200 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621205 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621210 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.621214 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.621802 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.630325 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.630341 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.631058 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.631066 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631121 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631126 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631129 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631133 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631135 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:44:24.633264 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631138 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631141 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631143 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631146 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631149 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631152 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631154 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631157 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631159 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631163 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631167 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631170 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631173 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631175 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631178 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631181 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631183 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631186 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631189 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:44:24.633626 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631192 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631194 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631197 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631199 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631202 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631206 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631209 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631212 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631215 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631217 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631219 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631222 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631224 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631227 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631229 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631232 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631234 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631237 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631239 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631242 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:44:24.634087 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631244 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631247 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631249 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631252 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631256 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631259 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631262 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631265 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631268 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631271 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631275 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631279 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631282 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631285 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631288 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631290 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631294 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631297 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631300 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:44:24.634645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631302 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631305 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631307 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631310 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631312 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631315 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631317 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631319 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631322 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631325 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631328 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631330 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631332 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631335 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631337 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631340 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631343 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631345 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631348 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631350 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:44:24.635132 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631352 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631355 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631357 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631359 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631362 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631364 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631367 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631370 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631373 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631376 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631378 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631381 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631383 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631386 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631388 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:44:24.635645 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631390 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.631396 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631509 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631514 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631517 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631520 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631522 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631525 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631528 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631531 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631535 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631538 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631541 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631543 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631547 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:44:24.636144 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631550 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631553 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631555 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631557 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631560 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631564 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631567 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631569 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631571 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631586 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631589 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631592 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631595 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631597 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631600 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631602 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631605 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631607 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631610 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631612 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:44:24.636507 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631615 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631617 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631620 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631622 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631625 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631628 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631631 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631635 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631637 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631641 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631644 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631646 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631650 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631653 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631655 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631658 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631660 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631663 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631665 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:44:24.637382 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631668 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631670 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631673 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631675 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631678 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631681 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631683 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631685 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631688 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631690 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631693 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631695 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631697 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631700 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631702 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631705 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631707 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631710 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631712 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631714 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:44:24.637911 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631717 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631719 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631722 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631724 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631726 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631729 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631731 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631734 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631736 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631739 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631741 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631744 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631747 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631749 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631752 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631754 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631757 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631759 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631762 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631764 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:44:24.638388 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631766 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631769 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631772 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631774 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631777 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631779 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:24.631782 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.631787 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.631893 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.633973 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.635611 2585 server.go:1019] "Starting client certificate rotation" Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.635722 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.635764 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:44:24.639031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.638825 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:44:24.639966 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.639941 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:44:24.654533 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.654511 2585 log.go:25] "Validated CRI v1 runtime API" Jul 07 15:44:24.658832 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.658817 2585 log.go:25] "Validated CRI v1 image API" Jul 07 15:44:24.660037 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.660023 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 15:44:24.663264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.663240 2585 fs.go:135] Filesystem UUIDs: map[50146896-afa6-4496-a8af-ec89ed396996:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 82df2dd3-068f-483d-a1ff-064e2874ec81:/dev/nvme0n1p3] Jul 07 15:44:24.663332 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.663263 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 15:44:24.669626 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.669463 2585 manager.go:216] Machine: {Timestamp:2026-07-07 15:44:24.668042431 +0000 UTC m=+0.350343428 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097915 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec216c53e3b60933b0cda31051d99d36 SystemUUID:ec216c53-e3b6-0933-b0cd-a31051d99d36 BootID:07255505-3852-4704-9b72-941e279eccff Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:64:32:8a:f0:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:64:32:8a:f0:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:94:ed:fd:cf:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 15:44:24.669626 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.669614 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 15:44:24.669761 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.669698 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 15:44:24.672461 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.672445 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:44:24.673657 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.673631 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 15:44:24.673793 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.673661 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-171.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 15:44:24.674429 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.674419 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 15:44:24.674465 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.674433 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 15:44:24.674465 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.674440 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:44:24.675124 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.675113 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:44:24.676770 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.676759 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 15:44:24.680128 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680118 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:44:24.680413 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680403 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 15:44:24.680445 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680436 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 07 15:44:24.680475 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680451 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 15:44:24.680475 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680467 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 15:44:24.680524 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680476 2585 kubelet.go:398] "Adding apiserver pod source" Jul 07 15:44:24.680524 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680485 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 15:44:24.680874 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680862 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:44:24.680917 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.680882 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:44:24.683006 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.682982 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 15:44:24.685652 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.685638 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 15:44:24.685697 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.685683 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 15:44:24.686914 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686903 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 15:44:24.686946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686919 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 15:44:24.686946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686925 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 15:44:24.686946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686930 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 15:44:24.686946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686939 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686954 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686962 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686973 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686980 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686985 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.686997 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 15:44:24.687047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.687002 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 15:44:24.688548 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.688535 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 15:44:24.688596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.688550 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 15:44:24.691739 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.691726 2585 server.go:1268] "Started kubelet" Jul 07 15:44:24.691848 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.691797 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 15:44:24.691908 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.691828 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 15:44:24.691908 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.691898 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 15:44:24.692745 ip-10-0-136-171 systemd[1]: Started Kubernetes Kubelet. Jul 07 15:44:24.693737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.693724 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 07 15:44:24.693963 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.693944 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-171.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 15:44:24.694026 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.693955 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-171.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 15:44:24.694136 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.694119 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 15:44:24.696090 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.696054 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 15:44:24.696492 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.696473 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 15:44:24.699731 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.698804 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-171.ec2.internal.18c00c615980ebb7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-171.ec2.internal,UID:ip-10-0-136-171.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-171.ec2.internal,},FirstTimestamp:2026-07-07 15:44:24.691698615 +0000 UTC m=+0.373999612,LastTimestamp:2026-07-07 15:44:24.691698615 +0000 UTC m=+0.373999612,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-171.ec2.internal,}" Jul 07 15:44:24.702330 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.702312 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 15:44:24.702330 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.702323 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 15:44:24.702486 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.702406 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:24.703115 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703058 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 15:44:24.703289 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703273 2585 factory.go:55] Registering systemd factory Jul 07 15:44:24.703394 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703377 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 15:44:24.703463 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703399 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 15:44:24.703515 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703498 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 07 15:44:24.703515 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703508 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 07 15:44:24.703645 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.703386 2585 factory.go:223] Registration of the systemd container factory successfully Jul 07 15:44:24.704255 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.704231 2585 factory.go:153] Registering CRI-O factory Jul 07 15:44:24.704255 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.704258 2585 factory.go:223] Registration of the crio container factory successfully Jul 07 15:44:24.704389 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.704313 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 15:44:24.704389 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.704338 2585 factory.go:103] Registering Raw factory Jul 07 15:44:24.704389 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.704363 2585 manager.go:1196] Started watching for new ooms in manager Jul 07 15:44:24.704837 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.704813 2585 manager.go:349] Starting recovery of all containers Jul 07 15:44:24.706677 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.706655 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hhntb" Jul 07 15:44:24.711034 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.711005 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-171.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 15:44:24.711034 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.711011 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 15:44:24.716647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.716519 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hhntb" Jul 07 15:44:24.716734 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.716566 2585 manager.go:354] Recovery completed Jul 07 15:44:24.720722 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.720710 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.722649 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.722633 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.722718 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.722668 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.722718 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.722681 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.723194 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723180 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 15:44:24.723194 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723193 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 15:44:24.723284 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723208 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:44:24.723284 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723228 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.723923 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723909 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.723993 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723937 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.723993 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.723951 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.724493 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.724481 2585 policy_none.go:49] "None policy: Start" Jul 07 15:44:24.724533 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.724496 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 15:44:24.724533 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.724506 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 15:44:24.725408 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.725341 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-171.ec2.internal.18c00c615b593274 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-171.ec2.internal,UID:ip-10-0-136-171.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-171.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-171.ec2.internal,},FirstTimestamp:2026-07-07 15:44:24.722649716 +0000 UTC m=+0.404950715,LastTimestamp:2026-07-07 15:44:24.722649716 +0000 UTC m=+0.404950715,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-171.ec2.internal,}" Jul 07 15:44:24.725676 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.725665 2585 policy_none.go:47] "Start" Jul 07 15:44:24.761110 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761092 2585 manager.go:337] "Starting Device Plugin manager" Jul 07 15:44:24.761187 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.761121 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 15:44:24.761187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761130 2585 server.go:87] "Starting device plugin registration server" Jul 07 15:44:24.761355 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761340 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 15:44:24.761438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761355 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 15:44:24.761482 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761466 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 15:44:24.761645 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761567 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 15:44:24.761645 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.761601 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 15:44:24.762050 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.762030 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 15:44:24.762121 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.762077 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:24.832258 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.832229 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 15:44:24.833610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.833561 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 15:44:24.833679 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.833613 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 15:44:24.833679 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.833634 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 15:44:24.833679 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.833664 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 15:44:24.836423 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.836407 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:44:24.861610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.861587 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.862369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.862325 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.862369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.862357 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.862369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.862368 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.862492 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.862390 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-171.ec2.internal" Jul 07 15:44:24.873454 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.873438 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-171.ec2.internal" Jul 07 15:44:24.873514 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.873457 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-171.ec2.internal\": node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:24.900524 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.900501 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:24.934565 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.934545 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal"] Jul 07 15:44:24.934666 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.934621 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.936025 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.936010 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.936097 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.936038 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.936097 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.936050 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.937345 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.937333 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.937460 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.937443 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" Jul 07 15:44:24.937503 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.937472 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.939785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.938244 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.939785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.938293 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.939785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.938307 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.939785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.938371 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.939785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.938406 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.939785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.938422 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.940293 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.940276 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:24.940362 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.940306 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:44:24.941570 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.941000 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:44:24.941570 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.941026 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:44:24.941570 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:24.941040 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:44:24.970108 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.970087 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-171.ec2.internal\" not found" node="ip-10-0-136-171.ec2.internal" Jul 07 15:44:24.974477 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:24.974460 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-171.ec2.internal\" not found" node="ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.001139 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.001118 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.005482 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.005467 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26ad4f4c872b4aa4b355d0fb0cf3a610-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal\" (UID: \"26ad4f4c872b4aa4b355d0fb0cf3a610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.005556 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.005492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26ad4f4c872b4aa4b355d0fb0cf3a610-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal\" (UID: \"26ad4f4c872b4aa4b355d0fb0cf3a610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.005556 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.005508 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72397af2de8d127b4272a28e33dc0313-config\") pod \"kube-apiserver-proxy-ip-10-0-136-171.ec2.internal\" (UID: \"72397af2de8d127b4272a28e33dc0313\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.102021 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.101996 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.106276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.106259 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72397af2de8d127b4272a28e33dc0313-config\") pod \"kube-apiserver-proxy-ip-10-0-136-171.ec2.internal\" (UID: \"72397af2de8d127b4272a28e33dc0313\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.106334 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.106283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26ad4f4c872b4aa4b355d0fb0cf3a610-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal\" (UID: \"26ad4f4c872b4aa4b355d0fb0cf3a610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.106334 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.106301 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26ad4f4c872b4aa4b355d0fb0cf3a610-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal\" (UID: \"26ad4f4c872b4aa4b355d0fb0cf3a610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.106401 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.106348 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26ad4f4c872b4aa4b355d0fb0cf3a610-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal\" (UID: \"26ad4f4c872b4aa4b355d0fb0cf3a610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.106401 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.106359 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72397af2de8d127b4272a28e33dc0313-config\") pod \"kube-apiserver-proxy-ip-10-0-136-171.ec2.internal\" (UID: \"72397af2de8d127b4272a28e33dc0313\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.106401 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.106388 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26ad4f4c872b4aa4b355d0fb0cf3a610-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal\" (UID: \"26ad4f4c872b4aa4b355d0fb0cf3a610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.202152 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.202065 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.273541 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.273500 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.278637 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.278620 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:25.302467 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.302443 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.402947 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.402913 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.503401 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.503324 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.603821 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.603789 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.635164 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.635132 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 15:44:25.635792 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.635308 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:44:25.650642 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.650619 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:44:25.704141 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.704111 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.719443 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.719411 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 15:39:25 +0000 UTC" deadline="2028-01-09 04:06:10.005286496 +0000 UTC" Jul 07 15:44:25.719443 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.719438 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13212h21m44.285851475s" Jul 07 15:44:25.764739 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.764665 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:44:25.804634 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.804604 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:25.827037 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:25.827006 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ad4f4c872b4aa4b355d0fb0cf3a610.slice/crio-860049b66a9b53a8ca70fa1a78c2666740af288b738f4cb38b5ad674b9264efd WatchSource:0}: Error finding container 860049b66a9b53a8ca70fa1a78c2666740af288b738f4cb38b5ad674b9264efd: Status 404 returned error can't find the container with id 860049b66a9b53a8ca70fa1a78c2666740af288b738f4cb38b5ad674b9264efd Jul 07 15:44:25.831174 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.831155 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:44:25.837542 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:25.837507 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" event={"ID":"26ad4f4c872b4aa4b355d0fb0cf3a610","Type":"ContainerStarted","Data":"860049b66a9b53a8ca70fa1a78c2666740af288b738f4cb38b5ad674b9264efd"} Jul 07 15:44:25.868919 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:25.868896 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72397af2de8d127b4272a28e33dc0313.slice/crio-e40cf442234608bf5bc6df8a68bb6cf8e8dc5d4f4bc51d0bbb647cc4484b911d WatchSource:0}: Error finding container e40cf442234608bf5bc6df8a68bb6cf8e8dc5d4f4bc51d0bbb647cc4484b911d: Status 404 returned error can't find the container with id e40cf442234608bf5bc6df8a68bb6cf8e8dc5d4f4bc51d0bbb647cc4484b911d Jul 07 15:44:25.905281 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:25.905256 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:26.005743 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:26.005721 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:27.296880 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:27.296852 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-171.ec2.internal\" not found" Jul 07 15:44:27.314172 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.314134 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:44:27.393907 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.393885 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" Jul 07 15:44:27.413351 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.413325 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:44:27.414325 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.414313 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" Jul 07 15:44:27.433422 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.433403 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:44:27.817137 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.817109 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:44:27.872685 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.872657 2585 apiserver.go:52] "Watching apiserver" Jul 07 15:44:27.882949 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.882918 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 15:44:27.883511 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.883487 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-wxh7c","openshift-network-diagnostics/network-check-target-7dkvf","kube-system/konnectivity-agent-wn9wc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k","openshift-cluster-node-tuning-operator/tuned-hjvlc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal","openshift-network-operator/iptables-alerter-dh2hc","openshift-ovn-kubernetes/ovnkube-node-7gmjg","kube-system/global-pull-secret-syncer-dl6lx","kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal","openshift-dns/node-resolver-z82v7","openshift-image-registry/node-ca-dxr9g","openshift-multus/multus-additional-cni-plugins-kqcj4","openshift-multus/multus-w99rv"] Jul 07 15:44:27.885268 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.885246 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:27.887755 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.886923 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.888867 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.888847 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:27.890118 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.890099 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.891253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.890985 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.891253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891013 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 15:44:27.891253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891103 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 15:44:27.891253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891183 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.891253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.890988 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.891493 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891289 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f2qrz\"" Jul 07 15:44:27.891765 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891747 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ggpxr\"" Jul 07 15:44:27.891765 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891757 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 15:44:27.892119 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.891985 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 15:44:27.892119 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.892055 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.892254 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.892145 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 15:44:27.892620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.892422 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 15:44:27.892743 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.892726 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.893072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.893054 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vptng\"" Jul 07 15:44:27.893973 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.893613 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 15:44:27.895049 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.895027 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 15:44:27.895297 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.895281 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vf65g\"" Jul 07 15:44:27.896735 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.896716 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.897236 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.897220 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:27.897330 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:27.897298 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:27.899026 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.898318 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:27.899026 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:27.898440 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:27.899964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.899879 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:27.899964 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:27.899933 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:27.900624 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.900606 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.900705 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.900634 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.900869 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.900843 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9hc66\"" Jul 07 15:44:27.901883 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.901866 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g6dfq\"" Jul 07 15:44:27.902092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.902070 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.902330 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.902317 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 15:44:27.902700 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.902682 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:27.902787 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.902745 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.904904 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905330 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-modprobe-d\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905353 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysctl-d\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905368 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-systemd\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-os-release\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905478 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ea2a280-4017-40a2-ae11-946fe06fe636-agent-certs\") pod \"konnectivity-agent-wn9wc\" (UID: \"8ea2a280-4017-40a2-ae11-946fe06fe636\") " pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905504 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2a280-4017-40a2-ae11-946fe06fe636-konnectivity-ca\") pod \"konnectivity-agent-wn9wc\" (UID: \"8ea2a280-4017-40a2-ae11-946fe06fe636\") " pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905527 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905669 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysctl-conf\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905703 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-run\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-tuned\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905754 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-netns\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905783 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905808 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-registration-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905831 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-host\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905889 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-cnibin\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.906321 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905917 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-socket-dir-parent\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905964 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-conf-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.905989 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-daemon-config\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906019 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-system-cni-dir\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906041 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8889\" (UniqueName: \"kubernetes.io/projected/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-kube-api-access-p8889\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906087 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-kubernetes\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906119 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-os-release\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-k8s-cni-cncf-io\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-etc-kubernetes\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/900391ac-b030-439b-81f2-46b1f809a97f-iptables-alerter-script\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906236 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cnibin\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906268 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-system-cni-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906307 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-cni-bin\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tql4v\" (UniqueName: \"kubernetes.io/projected/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-kube-api-access-tql4v\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906363 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906387 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysconfig\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.907040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906409 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-var-lib-kubelet\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-cni-multus\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906443 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-hostroot\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906462 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-device-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906490 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-sys\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906504 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-lib-modules\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906525 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-tmp\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906551 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-672rf\" (UniqueName: \"kubernetes.io/projected/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-kube-api-access-672rf\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906565 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-kubelet\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906579 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-socket-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906593 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-cni-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906612 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-cni-binary-copy\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906619 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-multus-certs\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/900391ac-b030-439b-81f2-46b1f809a97f-host-slash\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4skd\" (UniqueName: \"kubernetes.io/projected/900391ac-b030-439b-81f2-46b1f809a97f-kube-api-access-q4skd\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:27.908005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906721 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cni-binary-copy\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrgth\" (UniqueName: \"kubernetes.io/projected/02c1a6eb-643d-420a-abbb-6b5cd2091e70-kube-api-access-jrgth\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.906872 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-sys-fs\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.907645 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908114 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908245 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908302 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hswqs\"" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908435 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908450 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908594 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 15:44:27.908800 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908697 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.909367 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.908930 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.910340 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.910320 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 15:44:27.912215 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.912192 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v7mck\"" Jul 07 15:44:27.914883 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.914865 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 15:44:27.915132 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.915116 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:44:27.915250 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.915210 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nb2d2\"" Jul 07 15:44:27.916016 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.916000 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 15:44:27.956450 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.956425 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wtw7l" Jul 07 15:44:27.967845 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.967826 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wtw7l" Jul 07 15:44:27.994922 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:27.994854 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 15:44:28.007046 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007011 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.007144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007049 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysconfig\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007077 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-cni-multus\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.007144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-node-log\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.007144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007130 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-device-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007197 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-sys\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007211 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007228 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-cni-multus\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-lib-modules\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007269 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-tmp\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007280 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysconfig\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007296 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-cni-binary-copy\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.007336 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-multus-certs\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007340 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-lib-modules\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007346 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/900391ac-b030-439b-81f2-46b1f809a97f-host-slash\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007369 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/234a4877-fd36-4a8e-923e-24b3bee4b1bb-dbus\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007384 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-kubelet\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007386 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-sys\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007390 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-device-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007399 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/900391ac-b030-439b-81f2-46b1f809a97f-iptables-alerter-script\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007424 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cni-binary-copy\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007453 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-sys-fs\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-tuned\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-ovn\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007500 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71454b54-02a4-4252-8c45-3f5036ecffa0-ovn-node-metrics-cert\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-os-release\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007531 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ea2a280-4017-40a2-ae11-946fe06fe636-agent-certs\") pod \"konnectivity-agent-wn9wc\" (UID: \"8ea2a280-4017-40a2-ae11-946fe06fe636\") " pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007545 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2a280-4017-40a2-ae11-946fe06fe636-konnectivity-ca\") pod \"konnectivity-agent-wn9wc\" (UID: \"8ea2a280-4017-40a2-ae11-946fe06fe636\") " pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:28.007623 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007575 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-conf-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007589 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-systemd-units\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007602 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27198ddb-32a7-430e-a846-a61e18ba37e3-hosts-file\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88836d19-83c3-4e9f-bb58-e223acbc47d4-serviceca\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007651 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007666 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-cnibin\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007681 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007695 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-run-netns\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007710 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-system-cni-dir\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-k8s-cni-cncf-io\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007739 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6slcv\" (UniqueName: \"kubernetes.io/projected/24014c1a-3f58-4032-9847-98ddf8a3cec0-kube-api-access-6slcv\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-cni-bin\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007780 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-slash\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007794 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-var-lib-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007807 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-cni-bin\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007821 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-cni-netd\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.008574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007841 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-ovnkube-script-lib\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007856 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-var-lib-kubelet\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-hostroot\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007885 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/234a4877-fd36-4a8e-923e-24b3bee4b1bb-kubelet-config\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007899 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-672rf\" (UniqueName: \"kubernetes.io/projected/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-kube-api-access-672rf\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007920 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-cni-binary-copy\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007934 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-kubelet\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007933 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.007952 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4skd\" (UniqueName: \"kubernetes.io/projected/900391ac-b030-439b-81f2-46b1f809a97f-kube-api-access-q4skd\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008001 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-log-socket\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008006 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-multus-certs\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008055 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hszt\" (UniqueName: \"kubernetes.io/projected/71454b54-02a4-4252-8c45-3f5036ecffa0-kube-api-access-5hszt\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008094 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-socket-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008120 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-cni-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008146 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrgth\" (UniqueName: \"kubernetes.io/projected/02c1a6eb-643d-420a-abbb-6b5cd2091e70-kube-api-access-jrgth\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.009342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008192 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-modprobe-d\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008203 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-cnibin\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008251 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysctl-d\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008311 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-hostroot\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008337 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-modprobe-d\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008351 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-system-cni-dir\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008378 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/900391ac-b030-439b-81f2-46b1f809a97f-host-slash\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008401 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-kubelet\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008419 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-socket-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008441 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-var-lib-kubelet\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008467 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008480 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008502 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-k8s-cni-cncf-io\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008498 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008525 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-conf-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008544 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysctl-d\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008555 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008567 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-os-release\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.010480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008739 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-var-lib-cni-bin\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008757 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cni-binary-copy\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008831 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-cni-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008860 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-systemd\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008902 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-etc-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008948 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-systemd\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.008961 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysctl-conf\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009010 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-sys-fs\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009036 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-run\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009079 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2a280-4017-40a2-ae11-946fe06fe636-konnectivity-ca\") pod \"konnectivity-agent-wn9wc\" (UID: \"8ea2a280-4017-40a2-ae11-946fe06fe636\") " pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-netns\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009146 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-sysctl-conf\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009191 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-run\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-registration-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009280 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-host\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009311 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-socket-dir-parent\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.011731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-daemon-config\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009350 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/900391ac-b030-439b-81f2-46b1f809a97f-iptables-alerter-script\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009363 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009374 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-host\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009388 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009412 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-registration-dir\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009436 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88836d19-83c3-4e9f-bb58-e223acbc47d4-host\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8889\" (UniqueName: \"kubernetes.io/projected/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-kube-api-access-p8889\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-kubernetes\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-os-release\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009495 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-host-run-netns\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009509 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-etc-kubernetes\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009524 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-systemd\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27198ddb-32a7-430e-a846-a61e18ba37e3-tmp-dir\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009553 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbbh4\" (UniqueName: \"kubernetes.io/projected/27198ddb-32a7-430e-a846-a61e18ba37e3-kube-api-access-wbbh4\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009562 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-os-release\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009576 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pnrx\" (UniqueName: \"kubernetes.io/projected/88836d19-83c3-4e9f-bb58-e223acbc47d4-kube-api-access-7pnrx\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.012464 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009592 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cnibin\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-system-cni-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009607 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-socket-dir-parent\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009622 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tql4v\" (UniqueName: \"kubernetes.io/projected/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-kube-api-access-tql4v\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009636 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-ovnkube-config\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009652 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-env-overrides\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009702 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02c1a6eb-643d-420a-abbb-6b5cd2091e70-cnibin\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009755 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-multus-daemon-config\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009764 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-system-cni-dir\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009816 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-etc-kubernetes\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.009864 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-kubernetes\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.011851 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-tmp\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.011927 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-etc-tuned\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.013047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.012217 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ea2a280-4017-40a2-ae11-946fe06fe636-agent-certs\") pod \"konnectivity-agent-wn9wc\" (UID: \"8ea2a280-4017-40a2-ae11-946fe06fe636\") " pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:28.018822 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.018716 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-672rf\" (UniqueName: \"kubernetes.io/projected/b579d3a8-f88f-47fc-ad7a-6f7a957b777b-kube-api-access-672rf\") pod \"tuned-hjvlc\" (UID: \"b579d3a8-f88f-47fc-ad7a-6f7a957b777b\") " pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.019926 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.019894 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4skd\" (UniqueName: \"kubernetes.io/projected/900391ac-b030-439b-81f2-46b1f809a97f-kube-api-access-q4skd\") pod \"iptables-alerter-dh2hc\" (UID: \"900391ac-b030-439b-81f2-46b1f809a97f\") " pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.020838 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.020815 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8889\" (UniqueName: \"kubernetes.io/projected/ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4-kube-api-access-p8889\") pod \"aws-ebs-csi-driver-node-h9c8k\" (UID: \"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.021211 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.021191 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrgth\" (UniqueName: \"kubernetes.io/projected/02c1a6eb-643d-420a-abbb-6b5cd2091e70-kube-api-access-jrgth\") pod \"multus-additional-cni-plugins-kqcj4\" (UID: \"02c1a6eb-643d-420a-abbb-6b5cd2091e70\") " pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.021778 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.021691 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tql4v\" (UniqueName: \"kubernetes.io/projected/552f14ac-121f-48eb-ac57-2b52a4aa2d2f-kube-api-access-tql4v\") pod \"multus-w99rv\" (UID: \"552f14ac-121f-48eb-ac57-2b52a4aa2d2f\") " pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.033364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.033337 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" event={"ID":"72397af2de8d127b4272a28e33dc0313","Type":"ContainerStarted","Data":"e40cf442234608bf5bc6df8a68bb6cf8e8dc5d4f4bc51d0bbb647cc4484b911d"} Jul 07 15:44:28.110539 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110468 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:28.110539 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110510 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:28.110539 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110535 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88836d19-83c3-4e9f-bb58-e223acbc47d4-host\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110562 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-systemd\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27198ddb-32a7-430e-a846-a61e18ba37e3-tmp-dir\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.110588 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbbh4\" (UniqueName: \"kubernetes.io/projected/27198ddb-32a7-430e-a846-a61e18ba37e3-kube-api-access-wbbh4\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110632 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pnrx\" (UniqueName: \"kubernetes.io/projected/88836d19-83c3-4e9f-bb58-e223acbc47d4-kube-api-access-7pnrx\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.110660 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:28.610638587 +0000 UTC m=+3.102357048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110696 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-ovnkube-config\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110723 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-env-overrides\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110751 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-node-log\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110782 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/234a4877-fd36-4a8e-923e-24b3bee4b1bb-dbus\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-kubelet\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110836 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-ovn\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110858 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71454b54-02a4-4252-8c45-3f5036ecffa0-ovn-node-metrics-cert\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110866 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88836d19-83c3-4e9f-bb58-e223acbc47d4-host\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110887 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-systemd-units\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110907 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-systemd\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110911 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27198ddb-32a7-430e-a846-a61e18ba37e3-hosts-file\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110934 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88836d19-83c3-4e9f-bb58-e223acbc47d4-serviceca\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110961 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.110987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.110984 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-run-netns\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111010 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6slcv\" (UniqueName: \"kubernetes.io/projected/24014c1a-3f58-4032-9847-98ddf8a3cec0-kube-api-access-6slcv\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111033 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-slash\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111077 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-var-lib-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-cni-bin\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111123 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-cni-netd\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111147 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-ovnkube-script-lib\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111191 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/234a4877-fd36-4a8e-923e-24b3bee4b1bb-kubelet-config\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111216 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-log-socket\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111290 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111315 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hszt\" (UniqueName: \"kubernetes.io/projected/71454b54-02a4-4252-8c45-3f5036ecffa0-kube-api-access-5hszt\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111332 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-ovnkube-config\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-etc-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111398 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.111438 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111427 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-run-netns\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111489 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111522 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-node-log\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111628 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-slash\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111643 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/234a4877-fd36-4a8e-923e-24b3bee4b1bb-dbus\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111671 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-var-lib-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111702 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-cni-bin\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111737 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-cni-netd\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111771 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-ovn\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111807 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-kubelet\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111863 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88836d19-83c3-4e9f-bb58-e223acbc47d4-serviceca\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111934 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-systemd-units\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.111938 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111965 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.111996 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:44:28.611981197 +0000 UTC m=+3.103699640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:28.112015 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.112009 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/234a4877-fd36-4a8e-923e-24b3bee4b1bb-kubelet-config\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.112031 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-etc-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.112033 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-run-openvswitch\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111218 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27198ddb-32a7-430e-a846-a61e18ba37e3-tmp-dir\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.112062 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71454b54-02a4-4252-8c45-3f5036ecffa0-log-socket\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.111739 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27198ddb-32a7-430e-a846-a61e18ba37e3-hosts-file\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.112233 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-ovnkube-script-lib\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.112611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.112379 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71454b54-02a4-4252-8c45-3f5036ecffa0-env-overrides\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.114500 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.114479 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71454b54-02a4-4252-8c45-3f5036ecffa0-ovn-node-metrics-cert\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.121021 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.120996 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:28.121021 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.121016 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:28.121188 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.121028 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:28.121188 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.121080 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:44:28.621066061 +0000 UTC m=+3.112784514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:28.124624 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.124600 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbbh4\" (UniqueName: \"kubernetes.io/projected/27198ddb-32a7-430e-a846-a61e18ba37e3-kube-api-access-wbbh4\") pod \"node-resolver-z82v7\" (UID: \"27198ddb-32a7-430e-a846-a61e18ba37e3\") " pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.125447 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.125426 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pnrx\" (UniqueName: \"kubernetes.io/projected/88836d19-83c3-4e9f-bb58-e223acbc47d4-kube-api-access-7pnrx\") pod \"node-ca-dxr9g\" (UID: \"88836d19-83c3-4e9f-bb58-e223acbc47d4\") " pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.125806 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.125779 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hszt\" (UniqueName: \"kubernetes.io/projected/71454b54-02a4-4252-8c45-3f5036ecffa0-kube-api-access-5hszt\") pod \"ovnkube-node-7gmjg\" (UID: \"71454b54-02a4-4252-8c45-3f5036ecffa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.126237 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.126216 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6slcv\" (UniqueName: \"kubernetes.io/projected/24014c1a-3f58-4032-9847-98ddf8a3cec0-kube-api-access-6slcv\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:28.202761 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.202732 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dh2hc" Jul 07 15:44:28.218834 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.218807 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" Jul 07 15:44:28.228640 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.228620 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:28.234252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.234233 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" Jul 07 15:44:28.241867 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.241849 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" Jul 07 15:44:28.249609 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.249592 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w99rv" Jul 07 15:44:28.257386 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.257367 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:28.266072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.266056 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z82v7" Jul 07 15:44:28.272666 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.272651 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dxr9g" Jul 07 15:44:28.616055 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.616019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:28.616282 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.616103 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:28.616282 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.616185 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:28.616282 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.616221 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:28.616282 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.616253 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:29.61623586 +0000 UTC m=+4.107954305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:28.616282 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.616276 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:44:29.616261413 +0000 UTC m=+4.107979857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:28.717368 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.717340 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:28.717519 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.717479 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:28.717519 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.717498 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:28.717519 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.717510 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:28.717642 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:28.717569 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:44:29.717552614 +0000 UTC m=+4.209271055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:28.970138 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.970009 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:39:27 +0000 UTC" deadline="2028-03-11 20:56:04.405779064 +0000 UTC" Jul 07 15:44:28.970138 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:28.970084 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14717h11m35.435698238s" Jul 07 15:44:29.054426 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.054388 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c1a6eb_643d_420a_abbb_6b5cd2091e70.slice/crio-dbfdfc8e29b31e1be9e2592e4e8d06c21e66e31b7f8068af02ebcf88415ac9e5 WatchSource:0}: Error finding container dbfdfc8e29b31e1be9e2592e4e8d06c21e66e31b7f8068af02ebcf88415ac9e5: Status 404 returned error can't find the container with id dbfdfc8e29b31e1be9e2592e4e8d06c21e66e31b7f8068af02ebcf88415ac9e5 Jul 07 15:44:29.055278 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.055254 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71454b54_02a4_4252_8c45_3f5036ecffa0.slice/crio-433b417ef26eec45f82fdd7f693ebfa3689d8c36c8840b93d0ef8d13599dd1db WatchSource:0}: Error finding container 433b417ef26eec45f82fdd7f693ebfa3689d8c36c8840b93d0ef8d13599dd1db: Status 404 returned error can't find the container with id 433b417ef26eec45f82fdd7f693ebfa3689d8c36c8840b93d0ef8d13599dd1db Jul 07 15:44:29.056781 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.056479 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea2a280_4017_40a2_ae11_946fe06fe636.slice/crio-2666289aa4d979a10ee1cf267918fa9d2f85b34770f1154f2de8f332f77d5850 WatchSource:0}: Error finding container 2666289aa4d979a10ee1cf267918fa9d2f85b34770f1154f2de8f332f77d5850: Status 404 returned error can't find the container with id 2666289aa4d979a10ee1cf267918fa9d2f85b34770f1154f2de8f332f77d5850 Jul 07 15:44:29.058219 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.058114 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552f14ac_121f_48eb_ac57_2b52a4aa2d2f.slice/crio-39fbb3c361dc3d8a30706691e4f6648cc4466639f2e980fe4c66ffc9df0fb82a WatchSource:0}: Error finding container 39fbb3c361dc3d8a30706691e4f6648cc4466639f2e980fe4c66ffc9df0fb82a: Status 404 returned error can't find the container with id 39fbb3c361dc3d8a30706691e4f6648cc4466639f2e980fe4c66ffc9df0fb82a Jul 07 15:44:29.061273 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.061153 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900391ac_b030_439b_81f2_46b1f809a97f.slice/crio-a1e7d799b1695711135344e5cc1d342b5d413c48bc6fc8db0f2606ee7dcbc262 WatchSource:0}: Error finding container a1e7d799b1695711135344e5cc1d342b5d413c48bc6fc8db0f2606ee7dcbc262: Status 404 returned error can't find the container with id a1e7d799b1695711135344e5cc1d342b5d413c48bc6fc8db0f2606ee7dcbc262 Jul 07 15:44:29.062863 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.062814 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88836d19_83c3_4e9f_bb58_e223acbc47d4.slice/crio-ef506c4424c2bd2f71ea744d2db1c053815bfcb13792986f8d179b88b2442a18 WatchSource:0}: Error finding container ef506c4424c2bd2f71ea744d2db1c053815bfcb13792986f8d179b88b2442a18: Status 404 returned error can't find the container with id ef506c4424c2bd2f71ea744d2db1c053815bfcb13792986f8d179b88b2442a18 Jul 07 15:44:29.063020 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.062997 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb579d3a8_f88f_47fc_ad7a_6f7a957b777b.slice/crio-63acb1620a74022204431ddaf14b4e29cf3c5d414a0f588ae116cff8344bdac7 WatchSource:0}: Error finding container 63acb1620a74022204431ddaf14b4e29cf3c5d414a0f588ae116cff8344bdac7: Status 404 returned error can't find the container with id 63acb1620a74022204431ddaf14b4e29cf3c5d414a0f588ae116cff8344bdac7 Jul 07 15:44:29.064656 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.064570 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27198ddb_32a7_430e_a846_a61e18ba37e3.slice/crio-a5353439193014c2f5a3209c87569d4982492f4e824ea27d7b9f516f253713fb WatchSource:0}: Error finding container a5353439193014c2f5a3209c87569d4982492f4e824ea27d7b9f516f253713fb: Status 404 returned error can't find the container with id a5353439193014c2f5a3209c87569d4982492f4e824ea27d7b9f516f253713fb Jul 07 15:44:29.066810 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:44:29.066349 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7dbda9_5d17_43a0_a8ee_18337eb9c3d4.slice/crio-074e2b77c3b834c13cf6b9fe7353e8c2b011d1dfa22c0a552d4452a9d1f1b8d5 WatchSource:0}: Error finding container 074e2b77c3b834c13cf6b9fe7353e8c2b011d1dfa22c0a552d4452a9d1f1b8d5: Status 404 returned error can't find the container with id 074e2b77c3b834c13cf6b9fe7353e8c2b011d1dfa22c0a552d4452a9d1f1b8d5 Jul 07 15:44:29.627598 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:29.627295 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:29.627598 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.627590 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:29.627822 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.627666 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:44:31.627647543 +0000 UTC m=+6.119366010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:29.627822 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.627683 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:29.627822 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.627735 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:31.627720348 +0000 UTC m=+6.119438788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:29.627822 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:29.627593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:29.728361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:29.728322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:29.728546 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.728496 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:29.728546 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.728519 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:29.728546 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.728532 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:29.728705 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:29.728591 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:44:31.728573175 +0000 UTC m=+6.220291630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:29.970802 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:29.970711 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:39:27 +0000 UTC" deadline="2028-04-23 19:39:52.290025992 +0000 UTC" Jul 07 15:44:29.970802 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:29.970749 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15747h55m22.319282258s" Jul 07 15:44:30.027664 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.027636 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:30.027842 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:30.027760 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:30.028177 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.028145 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:30.028294 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:30.028272 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:30.028357 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.028336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:30.028417 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:30.028403 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:30.052246 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.052216 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w99rv" event={"ID":"552f14ac-121f-48eb-ac57-2b52a4aa2d2f","Type":"ContainerStarted","Data":"39fbb3c361dc3d8a30706691e4f6648cc4466639f2e980fe4c66ffc9df0fb82a"} Jul 07 15:44:30.059911 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.059883 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerStarted","Data":"dbfdfc8e29b31e1be9e2592e4e8d06c21e66e31b7f8068af02ebcf88415ac9e5"} Jul 07 15:44:30.062664 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.062634 2585 generic.go:356] "Generic (PLEG): container finished" podID="26ad4f4c872b4aa4b355d0fb0cf3a610" containerID="559a4cab3d1b16db1555b53dd216aab71eb5e1606ecb6ca84cb0ebcf892629ba" exitCode=0 Jul 07 15:44:30.062932 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.062908 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" event={"ID":"26ad4f4c872b4aa4b355d0fb0cf3a610","Type":"ContainerDied","Data":"559a4cab3d1b16db1555b53dd216aab71eb5e1606ecb6ca84cb0ebcf892629ba"} Jul 07 15:44:30.085542 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.085516 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" event={"ID":"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4","Type":"ContainerStarted","Data":"074e2b77c3b834c13cf6b9fe7353e8c2b011d1dfa22c0a552d4452a9d1f1b8d5"} Jul 07 15:44:30.094914 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.094883 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z82v7" event={"ID":"27198ddb-32a7-430e-a846-a61e18ba37e3","Type":"ContainerStarted","Data":"a5353439193014c2f5a3209c87569d4982492f4e824ea27d7b9f516f253713fb"} Jul 07 15:44:30.101187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.101140 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" event={"ID":"b579d3a8-f88f-47fc-ad7a-6f7a957b777b","Type":"ContainerStarted","Data":"63acb1620a74022204431ddaf14b4e29cf3c5d414a0f588ae116cff8344bdac7"} Jul 07 15:44:30.116346 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.116320 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"433b417ef26eec45f82fdd7f693ebfa3689d8c36c8840b93d0ef8d13599dd1db"} Jul 07 15:44:30.125306 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.125280 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wn9wc" event={"ID":"8ea2a280-4017-40a2-ae11-946fe06fe636","Type":"ContainerStarted","Data":"2666289aa4d979a10ee1cf267918fa9d2f85b34770f1154f2de8f332f77d5850"} Jul 07 15:44:30.149568 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.149541 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" event={"ID":"72397af2de8d127b4272a28e33dc0313","Type":"ContainerStarted","Data":"edd76cb57d729e695b50caa757f0be377df874915636a3e1637663f10783048a"} Jul 07 15:44:30.161358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.161327 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dxr9g" event={"ID":"88836d19-83c3-4e9f-bb58-e223acbc47d4","Type":"ContainerStarted","Data":"ef506c4424c2bd2f71ea744d2db1c053815bfcb13792986f8d179b88b2442a18"} Jul 07 15:44:30.172507 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:30.172419 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dh2hc" event={"ID":"900391ac-b030-439b-81f2-46b1f809a97f","Type":"ContainerStarted","Data":"a1e7d799b1695711135344e5cc1d342b5d413c48bc6fc8db0f2606ee7dcbc262"} Jul 07 15:44:31.189254 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:31.189219 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" event={"ID":"26ad4f4c872b4aa4b355d0fb0cf3a610","Type":"ContainerStarted","Data":"7da0881741e3de63e3f24ece547424afe11d5310f2d6174f60a3261151b79c06"} Jul 07 15:44:31.206688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:31.206633 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-171.ec2.internal" podStartSLOduration=4.20661472 podStartE2EDuration="4.20661472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:30.166788221 +0000 UTC m=+4.658506685" watchObservedRunningTime="2026-07-07 15:44:31.20661472 +0000 UTC m=+5.698333204" Jul 07 15:44:31.642824 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:31.642789 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:31.642992 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:31.642857 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:31.643048 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.643001 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:31.643107 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.643062 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:35.643043741 +0000 UTC m=+10.134762188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:31.643477 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.643458 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:31.643565 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.643517 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:44:35.643500625 +0000 UTC m=+10.135219090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:31.744888 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:31.744173 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:31.744888 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.744352 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:31.744888 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.744371 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:31.744888 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.744388 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:31.744888 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:31.744451 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:44:35.744431399 +0000 UTC m=+10.236149844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:32.027337 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:32.027296 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:32.027559 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:32.027450 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:32.027826 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:32.027803 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:32.027929 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:32.027882 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:32.028005 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:32.027929 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:32.028005 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:32.027990 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:34.028082 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:34.027986 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:34.028527 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:34.028198 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:34.028605 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:34.028583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:34.028810 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:34.028786 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:34.028916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:34.028849 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:34.028974 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:34.028941 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:35.675875 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:35.675833 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:35.676348 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:35.675916 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:35.676348 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.676057 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:35.676348 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.676117 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:43.67609855 +0000 UTC m=+18.167816994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:35.676500 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.676465 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:35.676551 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.676502 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:44:43.676490497 +0000 UTC m=+18.168208937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:35.776993 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:35.776959 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:35.777154 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.777110 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:35.777154 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.777124 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:35.777154 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.777133 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:35.777275 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:35.777187 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:44:43.777173994 +0000 UTC m=+18.268892448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:36.026148 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:36.026107 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:36.026148 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:36.026148 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:36.026374 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:36.026112 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:36.026374 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:36.026244 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:36.026374 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:36.026345 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:36.026526 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:36.026448 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:38.025737 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:38.025709 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:38.026125 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:38.025709 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:38.026125 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:38.025836 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:38.026125 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:38.025938 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:38.026125 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:38.025988 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:38.026125 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:38.026075 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:40.025515 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:40.025427 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:40.026029 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:40.025439 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:40.026029 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:40.025561 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:40.026029 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:40.025542 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:40.026029 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:40.025645 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:40.026029 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:40.025724 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:42.025564 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:42.025526 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:42.025564 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:42.025563 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:42.026047 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:42.025530 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:42.026047 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:42.025658 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:42.026047 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:42.025774 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:42.026047 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:42.025848 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:43.743370 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:43.743335 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:43.743817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:43.743395 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:43.743817 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.743502 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:43.743817 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.743513 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:43.743817 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.743548 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:44:59.743535937 +0000 UTC m=+34.235254381 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:43.743817 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.743560 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:59.743554665 +0000 UTC m=+34.235273106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:43.844768 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:43.844731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:43.844954 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.844936 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:43.845065 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.844961 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:43.845065 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.844970 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:43.845065 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:43.845028 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:44:59.845010481 +0000 UTC m=+34.336728924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:44.025540 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:44.025459 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:44.025540 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:44.025509 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:44.025749 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:44.025613 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:44.025749 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:44.025721 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:44.025860 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:44.025829 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:44.025945 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:44.025925 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:46.025486 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:46.025457 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:46.025930 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:46.025558 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:46.025930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:46.025910 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:46.026040 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:46.026008 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:46.026091 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:46.026054 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:46.026132 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:46.026112 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:48.025496 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.025346 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:48.025496 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.025368 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:48.025496 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.025346 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:48.025496 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:48.025459 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:48.025879 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:48.025514 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:48.025879 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:48.025593 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:48.225797 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.225515 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" event={"ID":"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4","Type":"ContainerStarted","Data":"69720cefee644b923bc6a8107f7f56ffe4b4f05276c5d6d04c700f16124f91ee"} Jul 07 15:44:48.227340 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.227315 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z82v7" event={"ID":"27198ddb-32a7-430e-a846-a61e18ba37e3","Type":"ContainerStarted","Data":"af0643d2639488657d1ef9468eed1b59bd58e4394d3fe01124c83f6928862308"} Jul 07 15:44:48.229116 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.229092 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" event={"ID":"b579d3a8-f88f-47fc-ad7a-6f7a957b777b","Type":"ContainerStarted","Data":"8ea7e639bc49bc09863a248d2361b7edf0b05ee4ba123db1bc19edd373e41aa1"} Jul 07 15:44:48.232076 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.232057 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"4f1dbc6b02ab166f7a6053db35c9869d7649ab26cebda61c17f4f0073ba2f976"} Jul 07 15:44:48.233745 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.233727 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wn9wc" event={"ID":"8ea2a280-4017-40a2-ae11-946fe06fe636","Type":"ContainerStarted","Data":"377db657cbe9a9cce1b920b21ea0f292f73cc667469e0f617d2a371a090a2255"} Jul 07 15:44:48.235521 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.235362 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dxr9g" event={"ID":"88836d19-83c3-4e9f-bb58-e223acbc47d4","Type":"ContainerStarted","Data":"a8523735405c7969c4e9bc9c05b1277b873b8e109cf93525553dc431d712290d"} Jul 07 15:44:48.237713 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.237693 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w99rv" event={"ID":"552f14ac-121f-48eb-ac57-2b52a4aa2d2f","Type":"ContainerStarted","Data":"3b946d5e9dea6a72be924137cb1ef63cd4343a3527e5cf46b58628fd55cd0815"} Jul 07 15:44:48.239244 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.239224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerStarted","Data":"52e9ee2a1b3d72a14be80bd9d8541e82026269789845ed423a1d2909381a5b83"} Jul 07 15:44:48.243594 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.243557 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z82v7" podStartSLOduration=3.360728237 podStartE2EDuration="22.243546107s" totalImagesPullingTime="18.88281787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:48.243430972 +0000 UTC m=+22.735149436" watchObservedRunningTime="2026-07-07 15:44:48.243546107 +0000 UTC m=+22.735264570" Jul 07 15:44:48.244049 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.244023 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-171.ec2.internal" podStartSLOduration=21.244016581 podStartE2EDuration="21.244016581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:31.207491405 +0000 UTC m=+5.699209869" watchObservedRunningTime="2026-07-07 15:44:48.244016581 +0000 UTC m=+22.735735044" Jul 07 15:44:48.281962 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.281922 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wn9wc" podStartSLOduration=11.478368824 podStartE2EDuration="22.281905846s" totalImagesPullingTime="10.803537022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:48.281436926 +0000 UTC m=+22.773155390" watchObservedRunningTime="2026-07-07 15:44:48.281905846 +0000 UTC m=+22.773624491" Jul 07 15:44:48.300310 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.300256 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hjvlc" podStartSLOduration=3.413642714 podStartE2EDuration="22.300235049s" totalImagesPullingTime="18.886592335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:48.299342072 +0000 UTC m=+22.791060548" watchObservedRunningTime="2026-07-07 15:44:48.300235049 +0000 UTC m=+22.791953560" Jul 07 15:44:48.321929 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.321891 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w99rv" podStartSLOduration=3.3731994 podStartE2EDuration="22.321876367s" totalImagesPullingTime="18.948676967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:48.321460723 +0000 UTC m=+22.813179216" watchObservedRunningTime="2026-07-07 15:44:48.321876367 +0000 UTC m=+22.813594829" Jul 07 15:44:48.337961 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:48.337758 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dxr9g" podStartSLOduration=3.45090466 podStartE2EDuration="22.337737873s" totalImagesPullingTime="18.886833213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:48.337706299 +0000 UTC m=+22.829424766" watchObservedRunningTime="2026-07-07 15:44:48.337737873 +0000 UTC m=+22.829456339" Jul 07 15:44:49.244378 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.244349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"a38acd616b1162ba6f653d47525e2ab1b84df51deb2937d73901a45eb9c9a3ae"} Jul 07 15:44:49.244860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.244442 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"58bd5744c3f1a6ea9027d64aa4fe1afe5383fd4bf8227f0d832e1b269e627910"} Jul 07 15:44:49.244860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.244474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"5c0e971f8ea71920add51ffe7e81a720cdd6618087287b12fadd7b245c223189"} Jul 07 15:44:49.244860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.244504 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"eb262baf42d52ce64bb1d483031b4ad5de3d1ff6b7768eb46721c53b45d8b0a3"} Jul 07 15:44:49.244860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.244531 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"3be14bf4856460633b03360b1288501936352eb90fa0e1c04e1fd478c504106d"} Jul 07 15:44:49.246233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.246209 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dh2hc" event={"ID":"900391ac-b030-439b-81f2-46b1f809a97f","Type":"ContainerStarted","Data":"2f862d80b42bc9184cbb27b053ef0426bcbf880ef347240b89c38dc1dce5a0db"} Jul 07 15:44:49.247499 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.247478 2585 generic.go:356] "Generic (PLEG): container finished" podID="02c1a6eb-643d-420a-abbb-6b5cd2091e70" containerID="52e9ee2a1b3d72a14be80bd9d8541e82026269789845ed423a1d2909381a5b83" exitCode=0 Jul 07 15:44:49.247606 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.247561 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerDied","Data":"52e9ee2a1b3d72a14be80bd9d8541e82026269789845ed423a1d2909381a5b83"} Jul 07 15:44:49.262136 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.262096 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dh2hc" podStartSLOduration=4.376678085 podStartE2EDuration="23.262083142s" totalImagesPullingTime="18.885405057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:49.26202866 +0000 UTC m=+23.753747122" watchObservedRunningTime="2026-07-07 15:44:49.262083142 +0000 UTC m=+23.753801604" Jul 07 15:44:49.537326 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.537298 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 15:44:49.965815 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.965689 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:44:49.537320381Z","UUID":"08ab2bb0-e981-4a39-9b87-f53c0a2d98b8","Handler":null,"Name":"","Endpoint":""} Jul 07 15:44:49.967133 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.967114 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 15:44:49.967268 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:49.967170 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 15:44:50.013799 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:50.013774 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z82v7_27198ddb-32a7-430e-a846-a61e18ba37e3/dns-node-resolver/0.log" Jul 07 15:44:50.024969 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:50.024950 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:50.024969 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:50.024963 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:50.025134 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:50.024964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:50.025134 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:50.025040 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:50.025238 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:50.025125 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:50.025280 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:50.025236 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:50.253177 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:50.253109 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" event={"ID":"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4","Type":"ContainerStarted","Data":"c2ed001adf64123cf45c840ee11f47ac9889f9f2cb0fa1b436369a988b250cea"} Jul 07 15:44:50.591036 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:50.590965 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dxr9g_88836d19-83c3-4e9f-bb58-e223acbc47d4/node-ca/0.log" Jul 07 15:44:51.257290 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:51.257250 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" event={"ID":"ac7dbda9-5d17-43a0-a8ee-18337eb9c3d4","Type":"ContainerStarted","Data":"180315e87b52073b4a7610436a86b814c54b9e22612dd0770e27c254edff8a14"} Jul 07 15:44:51.261324 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:51.261299 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"f75ff15bb3836ed4033a77ce22afdbd486c9aa4f88bd2814814812dc31e0786d"} Jul 07 15:44:51.277076 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:51.277022 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9c8k" podStartSLOduration=3.782641394 podStartE2EDuration="25.277007804s" totalImagesPullingTime="21.49436641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:51.2767363 +0000 UTC m=+25.768454761" watchObservedRunningTime="2026-07-07 15:44:51.277007804 +0000 UTC m=+25.768726265" Jul 07 15:44:52.025094 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:52.024856 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:52.025358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:52.024864 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:52.025358 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:52.025203 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:52.025358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:52.024970 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:52.025358 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:52.025269 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:52.025358 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:52.025323 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:52.729763 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:52.729734 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:52.730431 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:52.730407 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:53.269699 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:53.269671 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" event={"ID":"71454b54-02a4-4252-8c45-3f5036ecffa0","Type":"ContainerStarted","Data":"331868b2c678994d26452d1b28bd98b6a397abb3faee0b791c78d67cd572e7f6"} Jul 07 15:44:53.269972 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:53.269957 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:53.270098 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:53.269991 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:53.270490 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:53.270474 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wn9wc" Jul 07 15:44:53.304507 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:53.304468 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" podStartSLOduration=8.339220191 podStartE2EDuration="27.304456319s" totalImagesPullingTime="18.965236128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:53.300695535 +0000 UTC m=+27.792413995" watchObservedRunningTime="2026-07-07 15:44:53.304456319 +0000 UTC m=+27.796174763" Jul 07 15:44:54.024961 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.024739 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:54.025688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.024770 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:54.025688 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:54.025056 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:54.025688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.024837 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:54.025688 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:54.025110 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:54.025688 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:54.025191 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:54.272139 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.272103 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:54.272313 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.272187 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:54.287664 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.287643 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:54.287813 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:54.287733 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:44:56.026007 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:56.025971 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:56.026445 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:56.026066 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:56.026445 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:56.026152 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:56.026445 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:56.026265 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:56.026445 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:56.026293 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:56.026445 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:56.026354 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:56.277783 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:56.277706 2585 generic.go:356] "Generic (PLEG): container finished" podID="02c1a6eb-643d-420a-abbb-6b5cd2091e70" containerID="2206e5bb2f405c7f176e8558497b71463f641dbd86c39e4fb57dd411934d83da" exitCode=0 Jul 07 15:44:56.277924 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:56.277792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerDied","Data":"2206e5bb2f405c7f176e8558497b71463f641dbd86c39e4fb57dd411934d83da"} Jul 07 15:44:57.282073 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:57.282002 2585 generic.go:356] "Generic (PLEG): container finished" podID="02c1a6eb-643d-420a-abbb-6b5cd2091e70" containerID="e1488671a233e77f945c0a6b2cb6b9d3e965ea5e71d166a8a7f89d7700cda64a" exitCode=0 Jul 07 15:44:57.282425 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:57.282073 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerDied","Data":"e1488671a233e77f945c0a6b2cb6b9d3e965ea5e71d166a8a7f89d7700cda64a"} Jul 07 15:44:58.025120 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:58.025098 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:58.025284 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:58.025116 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:58.025284 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:58.025225 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:44:58.025361 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:58.025295 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:44:58.025361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:58.025327 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:58.025439 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:58.025416 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:44:58.287829 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:58.287800 2585 generic.go:356] "Generic (PLEG): container finished" podID="02c1a6eb-643d-420a-abbb-6b5cd2091e70" containerID="b56a615b1a585f76508fb733805f2f341252dcb5642c574494ccb1dea32e95ee" exitCode=0 Jul 07 15:44:58.288352 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:58.287863 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerDied","Data":"b56a615b1a585f76508fb733805f2f341252dcb5642c574494ccb1dea32e95ee"} Jul 07 15:44:59.764640 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:59.764602 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:44:59.765101 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:59.764689 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:44:59.765101 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.764763 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:59.765101 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.764789 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:59.765101 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.764832 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret podName:234a4877-fd36-4a8e-923e-24b3bee4b1bb nodeName:}" failed. No retries permitted until 2026-07-07 15:45:31.764817301 +0000 UTC m=+66.256535741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret") pod "global-pull-secret-syncer-dl6lx" (UID: "234a4877-fd36-4a8e-923e-24b3bee4b1bb") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:59.765101 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.764845 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs podName:24014c1a-3f58-4032-9847-98ddf8a3cec0 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:31.76483816 +0000 UTC m=+66.256556601 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs") pod "network-metrics-daemon-wxh7c" (UID: "24014c1a-3f58-4032-9847-98ddf8a3cec0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:59.865844 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:44:59.865810 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:44:59.865998 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.865938 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:59.865998 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.865953 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:59.865998 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.865962 2585 projected.go:196] Error preparing data for projected volume kube-api-access-n9l5b for pod openshift-network-diagnostics/network-check-target-7dkvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:59.866101 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:44:59.866006 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b podName:0c47ec45-4b0a-4a48-bf1e-08ba0838a96c nodeName:}" failed. No retries permitted until 2026-07-07 15:45:31.865994186 +0000 UTC m=+66.357712626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n9l5b" (UniqueName: "kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b") pod "network-check-target-7dkvf" (UID: "0c47ec45-4b0a-4a48-bf1e-08ba0838a96c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:45:00.025194 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:00.025093 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:00.025361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:00.025093 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:00.025361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:00.025103 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:00.025361 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:00.025236 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:00.025361 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:00.025326 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:00.025545 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:00.025379 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:02.025503 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:02.025464 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:02.025970 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:02.025464 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:02.025970 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:02.025629 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:02.025970 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:02.025464 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:02.025970 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:02.025687 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:02.025970 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:02.025735 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:04.025107 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:04.025066 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:04.025107 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:04.025092 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:04.025560 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:04.025071 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:04.025560 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:04.025205 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:04.025560 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:04.025308 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:04.025560 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:04.025363 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:04.305097 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:04.305023 2585 generic.go:356] "Generic (PLEG): container finished" podID="02c1a6eb-643d-420a-abbb-6b5cd2091e70" containerID="e92a0aa7e41b4255492840c51486165af95b8767156ae24651ba83e0fb4ce065" exitCode=0 Jul 07 15:45:04.305097 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:04.305064 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerDied","Data":"e92a0aa7e41b4255492840c51486165af95b8767156ae24651ba83e0fb4ce065"} Jul 07 15:45:05.310357 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:05.310322 2585 generic.go:356] "Generic (PLEG): container finished" podID="02c1a6eb-643d-420a-abbb-6b5cd2091e70" containerID="f9b0c48d646f8b9fb0db99371fc731fc17ebba71ec6b6e813baf978aacd5671f" exitCode=0 Jul 07 15:45:05.310760 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:05.310384 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerDied","Data":"f9b0c48d646f8b9fb0db99371fc731fc17ebba71ec6b6e813baf978aacd5671f"} Jul 07 15:45:06.027466 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:06.027434 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:06.027466 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:06.027468 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:06.027725 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:06.027517 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:06.027725 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:06.027537 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:06.027725 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:06.027626 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:06.027725 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:06.027708 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:06.315216 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:06.315126 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" event={"ID":"02c1a6eb-643d-420a-abbb-6b5cd2091e70","Type":"ContainerStarted","Data":"6b28824c270abfe975c1ab6113f4fdaea825a43d06a303157cafedc56821189b"} Jul 07 15:45:06.340757 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:06.340716 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kqcj4" podStartSLOduration=8.271570689 podStartE2EDuration="40.340702493s" totalImagesPullingTime="32.069131804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:06.339147351 +0000 UTC m=+40.830865812" watchObservedRunningTime="2026-07-07 15:45:06.340702493 +0000 UTC m=+40.832420953" Jul 07 15:45:08.025092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:08.025046 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:08.025092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:08.025095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:08.025597 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:08.025059 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:08.025597 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:08.025193 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:08.025597 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:08.025284 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:08.025597 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:08.025369 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:10.028328 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:10.028298 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:10.028702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:10.028302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:10.028702 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:10.028391 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:10.028702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:10.028302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:10.028702 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:10.028472 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:10.028702 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:10.028530 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:12.025404 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:12.025369 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:12.025404 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:12.025390 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:12.025815 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:12.025369 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:12.025815 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:12.025475 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:12.025815 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:12.025532 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:12.025815 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:12.025595 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:14.025065 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:14.025037 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:14.025543 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:14.025037 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:14.025543 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:14.025155 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:14.025543 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:14.025125 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:14.025543 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:14.025252 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:14.025543 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:14.025341 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:15.510546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:15.510516 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7dkvf"] Jul 07 15:45:15.511269 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:15.510613 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:15.511269 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:15.510695 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:15.516603 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:15.516576 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxh7c"] Jul 07 15:45:15.516747 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:15.516677 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:15.516814 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:15.516794 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:15.530474 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:15.530448 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dl6lx"] Jul 07 15:45:15.530618 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:15.530551 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:15.530676 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:15.530653 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:17.025444 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:17.025401 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:17.025914 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:17.025401 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:17.025914 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:17.025525 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxh7c" podUID="24014c1a-3f58-4032-9847-98ddf8a3cec0" Jul 07 15:45:17.025914 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:17.025401 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:17.025914 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:17.025595 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dl6lx" podUID="234a4877-fd36-4a8e-923e-24b3bee4b1bb" Jul 07 15:45:17.025914 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:17.025668 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dkvf" podUID="0c47ec45-4b0a-4a48-bf1e-08ba0838a96c" Jul 07 15:45:18.881212 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.881152 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-171.ec2.internal" event="NodeReady" Jul 07 15:45:18.881596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.881335 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 15:45:18.930964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.930923 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vnw9p"] Jul 07 15:45:18.952095 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.952058 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zscjv"] Jul 07 15:45:18.952273 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.952203 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:18.955003 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.954973 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 15:45:18.955191 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.954973 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j6trp\"" Jul 07 15:45:18.955191 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.954974 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 15:45:18.972327 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.972292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vnw9p"] Jul 07 15:45:18.972435 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.972334 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zscjv"] Jul 07 15:45:18.972435 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.972416 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:18.976301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.975769 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 15:45:18.976301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.975841 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 15:45:18.976301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.976108 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 15:45:18.976301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.976117 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsn42\"" Jul 07 15:45:18.976301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:18.976247 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 15:45:19.006756 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006715 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57b264b8-d138-46d3-ae42-45fb48a33dba-crio-socket\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.006756 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006756 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57b264b8-d138-46d3-ae42-45fb48a33dba-data-volume\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.006951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006856 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-metrics-tls\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.006951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57b264b8-d138-46d3-ae42-45fb48a33dba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.006951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006927 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57b264b8-d138-46d3-ae42-45fb48a33dba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.007118 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006965 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-tmp-dir\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.007118 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006981 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d28t4\" (UniqueName: \"kubernetes.io/projected/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-kube-api-access-d28t4\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.007118 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.006997 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjxz8\" (UniqueName: \"kubernetes.io/projected/57b264b8-d138-46d3-ae42-45fb48a33dba-kube-api-access-rjxz8\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.007118 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.007014 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-config-volume\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.024821 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.024788 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:19.024947 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.024888 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:19.024947 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.024901 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:19.028793 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.028774 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:45:19.028902 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.028829 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:45:19.028957 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.028907 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wcvdn\"" Jul 07 15:45:19.029031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.029018 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ctzps\"" Jul 07 15:45:19.029085 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.029071 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:45:19.029124 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.029108 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:45:19.031536 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.031517 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k6t8d"] Jul 07 15:45:19.052407 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.052385 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k6t8d"] Jul 07 15:45:19.052496 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.052466 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.055528 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.055511 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 15:45:19.055633 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.055546 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 15:45:19.055633 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.055579 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 15:45:19.055721 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.055656 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vwnd2\"" Jul 07 15:45:19.108408 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108369 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57b264b8-d138-46d3-ae42-45fb48a33dba-crio-socket\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.108408 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108406 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57b264b8-d138-46d3-ae42-45fb48a33dba-data-volume\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.108631 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-metrics-tls\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.108631 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108493 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjr26\" (UniqueName: \"kubernetes.io/projected/8e224e69-6838-49f7-b568-4cdb20c1a653-kube-api-access-sjr26\") pod \"ingress-canary-k6t8d\" (UID: \"8e224e69-6838-49f7-b568-4cdb20c1a653\") " pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.108631 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108519 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57b264b8-d138-46d3-ae42-45fb48a33dba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.108631 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108546 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57b264b8-d138-46d3-ae42-45fb48a33dba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.108631 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108616 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57b264b8-d138-46d3-ae42-45fb48a33dba-crio-socket\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.108859 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108631 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-tmp-dir\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.108859 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108665 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d28t4\" (UniqueName: \"kubernetes.io/projected/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-kube-api-access-d28t4\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.108859 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108709 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjxz8\" (UniqueName: \"kubernetes.io/projected/57b264b8-d138-46d3-ae42-45fb48a33dba-kube-api-access-rjxz8\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.108859 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e224e69-6838-49f7-b568-4cdb20c1a653-cert\") pod \"ingress-canary-k6t8d\" (UID: \"8e224e69-6838-49f7-b568-4cdb20c1a653\") " pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.108859 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108769 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-config-volume\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.109086 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108884 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57b264b8-d138-46d3-ae42-45fb48a33dba-data-volume\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.109199 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.108965 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-tmp-dir\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.109307 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.109289 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57b264b8-d138-46d3-ae42-45fb48a33dba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.109352 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.109299 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-config-volume\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.112647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.112620 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57b264b8-d138-46d3-ae42-45fb48a33dba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.116471 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.116448 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-metrics-tls\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.118576 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.118553 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d28t4\" (UniqueName: \"kubernetes.io/projected/983ffad6-c30c-4d1b-9d7c-2c9c611f7f97-kube-api-access-d28t4\") pod \"dns-default-vnw9p\" (UID: \"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97\") " pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.118930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.118911 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjxz8\" (UniqueName: \"kubernetes.io/projected/57b264b8-d138-46d3-ae42-45fb48a33dba-kube-api-access-rjxz8\") pod \"insights-runtime-extractor-zscjv\" (UID: \"57b264b8-d138-46d3-ae42-45fb48a33dba\") " pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.210132 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.210022 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjr26\" (UniqueName: \"kubernetes.io/projected/8e224e69-6838-49f7-b568-4cdb20c1a653-kube-api-access-sjr26\") pod \"ingress-canary-k6t8d\" (UID: \"8e224e69-6838-49f7-b568-4cdb20c1a653\") " pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.210132 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.210083 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e224e69-6838-49f7-b568-4cdb20c1a653-cert\") pod \"ingress-canary-k6t8d\" (UID: \"8e224e69-6838-49f7-b568-4cdb20c1a653\") " pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.212610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.212581 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e224e69-6838-49f7-b568-4cdb20c1a653-cert\") pod \"ingress-canary-k6t8d\" (UID: \"8e224e69-6838-49f7-b568-4cdb20c1a653\") " pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.219215 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.219192 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjr26\" (UniqueName: \"kubernetes.io/projected/8e224e69-6838-49f7-b568-4cdb20c1a653-kube-api-access-sjr26\") pod \"ingress-canary-k6t8d\" (UID: \"8e224e69-6838-49f7-b568-4cdb20c1a653\") " pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.262065 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.262028 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:19.282237 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.282205 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zscjv" Jul 07 15:45:19.360626 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.360600 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k6t8d" Jul 07 15:45:19.457929 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.457895 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vnw9p"] Jul 07 15:45:19.465376 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.465351 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zscjv"] Jul 07 15:45:19.468469 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:19.468445 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b264b8_d138_46d3_ae42_45fb48a33dba.slice/crio-e4a0ecb561852e16b0c48b96717d6791c197c095627cee494352d173c8c91df7 WatchSource:0}: Error finding container e4a0ecb561852e16b0c48b96717d6791c197c095627cee494352d173c8c91df7: Status 404 returned error can't find the container with id e4a0ecb561852e16b0c48b96717d6791c197c095627cee494352d173c8c91df7 Jul 07 15:45:19.538466 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.538435 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k6t8d"] Jul 07 15:45:19.542015 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:19.541986 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e224e69_6838_49f7_b568_4cdb20c1a653.slice/crio-b2768edc56e0476626415730b444d42ad6c543b52ab6ef95cbbd95951e471745 WatchSource:0}: Error finding container b2768edc56e0476626415730b444d42ad6c543b52ab6ef95cbbd95951e471745: Status 404 returned error can't find the container with id b2768edc56e0476626415730b444d42ad6c543b52ab6ef95cbbd95951e471745 Jul 07 15:45:19.720391 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.720318 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv"] Jul 07 15:45:19.733004 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.732982 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.736395 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.736367 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv"] Jul 07 15:45:19.737085 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.736866 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 15:45:19.737085 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.736895 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 15:45:19.737085 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.736936 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 15:45:19.737085 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.736987 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 15:45:19.737085 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.736936 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-llg97\"" Jul 07 15:45:19.737388 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.737116 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 15:45:19.814815 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.814778 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqz27\" (UniqueName: \"kubernetes.io/projected/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-kube-api-access-sqz27\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.815009 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.814822 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.815009 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.814844 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.815009 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.814862 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.915574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.915535 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqz27\" (UniqueName: \"kubernetes.io/projected/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-kube-api-access-sqz27\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.915574 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.915577 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.916051 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.915598 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.916051 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.915620 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.916455 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.916431 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.918781 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.918761 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.918839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.918767 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:19.924785 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:19.924763 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqz27\" (UniqueName: \"kubernetes.io/projected/cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06-kube-api-access-sqz27\") pod \"prometheus-operator-567f7b8b7c-fcqxv\" (UID: \"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:20.043424 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:20.043395 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" Jul 07 15:45:20.228521 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:20.228483 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv"] Jul 07 15:45:20.237098 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:20.237040 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9c9816_4c5b_49fa_a4ec_ec7fa1826c06.slice/crio-1ac3e8a55b43c7065a01b413e02821caec5aaaa3b4dc972416f9ff2c2fca4dc8 WatchSource:0}: Error finding container 1ac3e8a55b43c7065a01b413e02821caec5aaaa3b4dc972416f9ff2c2fca4dc8: Status 404 returned error can't find the container with id 1ac3e8a55b43c7065a01b413e02821caec5aaaa3b4dc972416f9ff2c2fca4dc8 Jul 07 15:45:20.358541 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:20.358466 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zscjv" event={"ID":"57b264b8-d138-46d3-ae42-45fb48a33dba","Type":"ContainerStarted","Data":"e4a0ecb561852e16b0c48b96717d6791c197c095627cee494352d173c8c91df7"} Jul 07 15:45:20.362605 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:20.362572 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" event={"ID":"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06","Type":"ContainerStarted","Data":"1ac3e8a55b43c7065a01b413e02821caec5aaaa3b4dc972416f9ff2c2fca4dc8"} Jul 07 15:45:20.364151 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:20.364123 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vnw9p" event={"ID":"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97","Type":"ContainerStarted","Data":"8eeb4f48d9a07544cbff8eb39479f2a285cc789408dbc389f84f9359482947d7"} Jul 07 15:45:20.365698 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:20.365647 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k6t8d" event={"ID":"8e224e69-6838-49f7-b568-4cdb20c1a653","Type":"ContainerStarted","Data":"b2768edc56e0476626415730b444d42ad6c543b52ab6ef95cbbd95951e471745"} Jul 07 15:45:25.389870 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.389619 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" event={"ID":"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06","Type":"ContainerStarted","Data":"fe3c311e46f0c6ad7e5399fd56638f531dfc731971926b5286fde411517c2170"} Jul 07 15:45:25.390717 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.389935 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" event={"ID":"cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06","Type":"ContainerStarted","Data":"b94654c1ad13076f5d37b02ec9397a0a8550436816af9c5e6b2083e2ecc072cc"} Jul 07 15:45:25.392238 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.392208 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vnw9p" event={"ID":"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97","Type":"ContainerStarted","Data":"08c60a2978e099ed8dde7f09dac115016269f0a289271f1cb609c2e123dd083d"} Jul 07 15:45:25.392362 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.392251 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vnw9p" event={"ID":"983ffad6-c30c-4d1b-9d7c-2c9c611f7f97","Type":"ContainerStarted","Data":"0a4148956a4c9570349dc7160390edc1526b4da65aba1b2422d7109da45d4bfa"} Jul 07 15:45:25.394307 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.393430 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:25.396988 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.396570 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k6t8d" event={"ID":"8e224e69-6838-49f7-b568-4cdb20c1a653","Type":"ContainerStarted","Data":"61743684270b025d09d89c8511900fafafe7826cef8640976301dc803d47c626"} Jul 07 15:45:25.398672 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.398603 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zscjv" event={"ID":"57b264b8-d138-46d3-ae42-45fb48a33dba","Type":"ContainerStarted","Data":"36cf0f277eb1f35a01bf523f42f0545773200a4877c6cff61c105b7e7f065ccb"} Jul 07 15:45:25.409079 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.409031 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcqxv" podStartSLOduration=1.587888053 podStartE2EDuration="6.409015657s" totalImagesPullingTime="4.821127604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:25.408146398 +0000 UTC m=+59.899864862" watchObservedRunningTime="2026-07-07 15:45:25.409015657 +0000 UTC m=+59.900734098" Jul 07 15:45:25.427033 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.426948 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vnw9p" podStartSLOduration=1.83255061 podStartE2EDuration="7.426932429s" totalImagesPullingTime="5.594381819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:25.426528155 +0000 UTC m=+59.918246619" watchObservedRunningTime="2026-07-07 15:45:25.426932429 +0000 UTC m=+59.918650892" Jul 07 15:45:25.443815 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:25.443776 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k6t8d" podStartSLOduration=0.902251028 podStartE2EDuration="6.443762575s" totalImagesPullingTime="5.541511547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:25.443373768 +0000 UTC m=+59.935092231" watchObservedRunningTime="2026-07-07 15:45:25.443762575 +0000 UTC m=+59.935481201" Jul 07 15:45:26.293091 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:26.292920 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gmjg" Jul 07 15:45:26.404194 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:26.404131 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zscjv" event={"ID":"57b264b8-d138-46d3-ae42-45fb48a33dba","Type":"ContainerStarted","Data":"92d4a3379b55f27924fca9f5d489303f3909069776399437743b47f1e0613c2f"} Jul 07 15:45:27.348406 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.348359 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp"] Jul 07 15:45:27.364892 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.364806 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r5bzj"] Jul 07 15:45:27.365976 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.365481 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.368798 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.368771 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 15:45:27.370188 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.370142 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 15:45:27.370303 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.370245 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-26jxk\"" Jul 07 15:45:27.372249 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.372230 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8"] Jul 07 15:45:27.374060 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.374034 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.375887 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.375864 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp"] Jul 07 15:45:27.375968 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.375944 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.376625 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.376604 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 15:45:27.377016 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.376852 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l97nq\"" Jul 07 15:45:27.377016 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.376948 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 15:45:27.378742 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.378395 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 15:45:27.379801 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.379780 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 15:45:27.379908 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.379787 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 15:45:27.379908 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.379828 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 15:45:27.380358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.380343 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t4q85\"" Jul 07 15:45:27.400841 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.400764 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8"] Jul 07 15:45:27.482176 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482135 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482192 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482229 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-textfile\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482249 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwl6r\" (UniqueName: \"kubernetes.io/projected/29156cfa-8705-442d-9ee7-32719a333b8c-kube-api-access-vwl6r\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482362 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/59b448ea-38f8-4aec-aa8a-d379b07accff-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482398 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-wtmp\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482437 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-sys\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482547 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482605 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-metrics-client-ca\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482648 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6zx5\" (UniqueName: \"kubernetes.io/projected/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-kube-api-access-f6zx5\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482702 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wm7v\" (UniqueName: \"kubernetes.io/projected/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-api-access-9wm7v\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482780 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482820 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59b448ea-38f8-4aec-aa8a-d379b07accff-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482841 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29156cfa-8705-442d-9ee7-32719a333b8c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482867 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-root\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.482994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.482888 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-tls\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.583496 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583463 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6zx5\" (UniqueName: \"kubernetes.io/projected/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-kube-api-access-f6zx5\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.583496 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583502 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583521 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wm7v\" (UniqueName: \"kubernetes.io/projected/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-api-access-9wm7v\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583604 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59b448ea-38f8-4aec-aa8a-d379b07accff-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583633 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29156cfa-8705-442d-9ee7-32719a333b8c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583659 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-root\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.583720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583683 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-tls\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583737 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583761 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583783 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-root\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.583805 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-textfile\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:27.583890 2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:27.583948 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-tls podName:29156cfa-8705-442d-9ee7-32719a333b8c nodeName:}" failed. No retries permitted until 2026-07-07 15:45:28.083929377 +0000 UTC m=+62.575647821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-nz7cp" (UID: "29156cfa-8705-442d-9ee7-32719a333b8c") : secret "openshift-state-metrics-tls" not found Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584126 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-textfile\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:27.584193 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwl6r\" (UniqueName: \"kubernetes.io/projected/29156cfa-8705-442d-9ee7-32719a333b8c-kube-api-access-vwl6r\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:27.584243 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-tls podName:6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:28.084228686 +0000 UTC m=+62.575947128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-tls") pod "node-exporter-r5bzj" (UID: "6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3") : secret "node-exporter-tls" not found Jul 07 15:45:27.584276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/59b448ea-38f8-4aec-aa8a-d379b07accff-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-wtmp\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584349 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-sys\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584376 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584403 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-metrics-client-ca\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584425 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584498 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59b448ea-38f8-4aec-aa8a-d379b07accff-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584501 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29156cfa-8705-442d-9ee7-32719a333b8c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584529 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-wtmp\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584538 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.584723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584588 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-sys\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.585110 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.584751 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/59b448ea-38f8-4aec-aa8a-d379b07accff-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.585110 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.585053 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.585257 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.585112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-metrics-client-ca\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.587968 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.587933 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.588079 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.588044 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.588151 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.588125 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.588151 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.588125 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.596604 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.596581 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwl6r\" (UniqueName: \"kubernetes.io/projected/29156cfa-8705-442d-9ee7-32719a333b8c-kube-api-access-vwl6r\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:27.596716 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.596644 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wm7v\" (UniqueName: \"kubernetes.io/projected/59b448ea-38f8-4aec-aa8a-d379b07accff-kube-api-access-9wm7v\") pod \"kube-state-metrics-7c9dbb8b7d-nspm8\" (UID: \"59b448ea-38f8-4aec-aa8a-d379b07accff\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:27.596992 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.596973 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6zx5\" (UniqueName: \"kubernetes.io/projected/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-kube-api-access-f6zx5\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:27.699657 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:27.699578 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" Jul 07 15:45:28.089840 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.089802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-tls\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:28.090039 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.089872 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:28.092542 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.092511 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3-node-exporter-tls\") pod \"node-exporter-r5bzj\" (UID: \"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3\") " pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:28.092679 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.092661 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29156cfa-8705-442d-9ee7-32719a333b8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nz7cp\" (UID: \"29156cfa-8705-442d-9ee7-32719a333b8c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:28.280813 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.280778 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:45:28.283695 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.283672 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" Jul 07 15:45:28.291709 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.291682 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.291869 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.291718 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5bzj" Jul 07 15:45:28.294556 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.294537 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 15:45:28.294656 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.294561 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 15:45:28.294863 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.294845 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 15:45:28.294981 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.294904 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 15:45:28.295035 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.295015 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 15:45:28.295319 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.295299 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 15:45:28.295744 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.295724 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 15:45:28.295861 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.295844 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zkfx4\"" Jul 07 15:45:28.295906 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.295866 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 15:45:28.296181 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.296147 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 15:45:28.301462 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.301379 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:45:28.392435 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392354 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392435 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392405 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392660 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392483 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27289\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-kube-api-access-27289\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392660 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392514 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392660 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392549 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392660 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392577 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-web-config\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392660 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392599 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-tls-assets\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392718 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-config-volume\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392741 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.392839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392802 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-config-out\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.393029 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.392844 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.493877 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.493846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.493877 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.493880 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494059 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27289\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-kube-api-access-27289\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494184 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-web-config\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494212 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494248 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-tls-assets\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494276 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-config-volume\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494297 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494327 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-config-out\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.494392 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:28.494372 2585 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 15:45:28.495009 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.494416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.495009 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:28.494431 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls podName:855d73b5-0aec-4fca-929b-9ba10bb89881 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:28.994410178 +0000 UTC m=+63.486128625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881") : secret "alertmanager-main-tls" not found Jul 07 15:45:28.495552 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.495526 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.495747 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.495721 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.495816 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.495755 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.497184 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.497134 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.497635 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.497605 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.498670 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.498298 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-config-out\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.498670 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.498459 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.498670 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.498602 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-tls-assets\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.498670 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.498628 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.499642 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.499620 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-config-volume\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.500054 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.500029 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-web-config\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.506092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.506064 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27289\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-kube-api-access-27289\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:28.705727 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:28.705699 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f2e3e28_4afc_41b5_a6d0_32fd6558c8a3.slice/crio-f90bce11c48befb6abcb9516fd9d9c5109d64ee0d91734ea9eb5e30ca6c8d486 WatchSource:0}: Error finding container f90bce11c48befb6abcb9516fd9d9c5109d64ee0d91734ea9eb5e30ca6c8d486: Status 404 returned error can't find the container with id f90bce11c48befb6abcb9516fd9d9c5109d64ee0d91734ea9eb5e30ca6c8d486 Jul 07 15:45:28.847184 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.847134 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8"] Jul 07 15:45:28.849721 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:28.849695 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b448ea_38f8_4aec_aa8a_d379b07accff.slice/crio-d084fe7eb30a591d1fc3303479942eef14dffdb0ca50c9b93ceee7f614de96fa WatchSource:0}: Error finding container d084fe7eb30a591d1fc3303479942eef14dffdb0ca50c9b93ceee7f614de96fa: Status 404 returned error can't find the container with id d084fe7eb30a591d1fc3303479942eef14dffdb0ca50c9b93ceee7f614de96fa Jul 07 15:45:28.864460 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.864430 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp"] Jul 07 15:45:28.867528 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:28.867502 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29156cfa_8705_442d_9ee7_32719a333b8c.slice/crio-e1ea6417592da0cc5995330618b106246694f5c5a376a067c0ee884a132dfb25 WatchSource:0}: Error finding container e1ea6417592da0cc5995330618b106246694f5c5a376a067c0ee884a132dfb25: Status 404 returned error can't find the container with id e1ea6417592da0cc5995330618b106246694f5c5a376a067c0ee884a132dfb25 Jul 07 15:45:28.999114 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:28.999083 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:29.002072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.002044 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:29.203830 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.203788 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:45:29.371717 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.371615 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:45:29.376883 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:29.376856 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855d73b5_0aec_4fca_929b_9ba10bb89881.slice/crio-9b40dccf30dbcad17017ffb9e19756943254a4359efd8362f005895b77ddb760 WatchSource:0}: Error finding container 9b40dccf30dbcad17017ffb9e19756943254a4359efd8362f005895b77ddb760: Status 404 returned error can't find the container with id 9b40dccf30dbcad17017ffb9e19756943254a4359efd8362f005895b77ddb760 Jul 07 15:45:29.388263 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.385916 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d68f8b574-62sm9"] Jul 07 15:45:29.391380 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.391346 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.394257 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394234 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 15:45:29.394495 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394326 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 15:45:29.394581 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394500 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 15:45:29.394581 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394433 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dhcrqbp4umaoa\"" Jul 07 15:45:29.394702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394367 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 15:45:29.394702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394448 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 15:45:29.394702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.394426 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-f9jzd\"" Jul 07 15:45:29.401986 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.401966 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d68f8b574-62sm9"] Jul 07 15:45:29.415945 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.415912 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" event={"ID":"59b448ea-38f8-4aec-aa8a-d379b07accff","Type":"ContainerStarted","Data":"d084fe7eb30a591d1fc3303479942eef14dffdb0ca50c9b93ceee7f614de96fa"} Jul 07 15:45:29.419779 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.419682 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zscjv" event={"ID":"57b264b8-d138-46d3-ae42-45fb48a33dba","Type":"ContainerStarted","Data":"68dd53de211fff9fd996a7c55409595f79bfc2889e3741d8859134a5479de47b"} Jul 07 15:45:29.421360 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.421272 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"9b40dccf30dbcad17017ffb9e19756943254a4359efd8362f005895b77ddb760"} Jul 07 15:45:29.424237 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.424198 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" event={"ID":"29156cfa-8705-442d-9ee7-32719a333b8c","Type":"ContainerStarted","Data":"edb2d56aec75ec112a6a919a3dab6f03c1de5c541559847d37b89c6a14f5c496"} Jul 07 15:45:29.424366 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.424273 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" event={"ID":"29156cfa-8705-442d-9ee7-32719a333b8c","Type":"ContainerStarted","Data":"c8cc96759d0df024f87f6b19b0b41f089e1d7dc362f62e3be9503cfdb13d361c"} Jul 07 15:45:29.424366 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.424302 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" event={"ID":"29156cfa-8705-442d-9ee7-32719a333b8c","Type":"ContainerStarted","Data":"e1ea6417592da0cc5995330618b106246694f5c5a376a067c0ee884a132dfb25"} Jul 07 15:45:29.425675 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.425651 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5bzj" event={"ID":"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3","Type":"ContainerStarted","Data":"f90bce11c48befb6abcb9516fd9d9c5109d64ee0d91734ea9eb5e30ca6c8d486"} Jul 07 15:45:29.442012 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.441958 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zscjv" podStartSLOduration=2.407744108 podStartE2EDuration="11.441936786s" totalImagesPullingTime="9.034192678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:45:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:29.44141318 +0000 UTC m=+63.933131895" watchObservedRunningTime="2026-07-07 15:45:29.441936786 +0000 UTC m=+63.933655249" Jul 07 15:45:29.503753 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503711 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxwg4\" (UniqueName: \"kubernetes.io/projected/4e6258fa-5b77-4809-a472-fe0f23960f80-kube-api-access-lxwg4\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503856 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e6258fa-5b77-4809-a472-fe0f23960f80-metrics-client-ca\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503879 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-grpc-tls\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-tls\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.503987 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.504146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.504026 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605206 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605152 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605380 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605237 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605380 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605281 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605380 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605308 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxwg4\" (UniqueName: \"kubernetes.io/projected/4e6258fa-5b77-4809-a472-fe0f23960f80-kube-api-access-lxwg4\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605616 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605382 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e6258fa-5b77-4809-a472-fe0f23960f80-metrics-client-ca\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605616 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605407 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-grpc-tls\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605616 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-tls\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.605616 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.605495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.607048 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.607017 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e6258fa-5b77-4809-a472-fe0f23960f80-metrics-client-ca\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.617449 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.617415 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxwg4\" (UniqueName: \"kubernetes.io/projected/4e6258fa-5b77-4809-a472-fe0f23960f80-kube-api-access-lxwg4\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.622532 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.622499 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.622682 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.622543 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.623131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.623105 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-grpc-tls\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.623566 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.623539 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.623987 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.623961 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.626815 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.626754 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4e6258fa-5b77-4809-a472-fe0f23960f80-secret-thanos-querier-tls\") pod \"thanos-querier-5d68f8b574-62sm9\" (UID: \"4e6258fa-5b77-4809-a472-fe0f23960f80\") " pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.702958 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.702916 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:29.881375 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:29.881328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d68f8b574-62sm9"] Jul 07 15:45:30.042385 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:30.042290 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6258fa_5b77_4809_a472_fe0f23960f80.slice/crio-96f8c20e91d10accc968dc919e9971fb8ca255f33f8e41dab6da5e828b42b697 WatchSource:0}: Error finding container 96f8c20e91d10accc968dc919e9971fb8ca255f33f8e41dab6da5e828b42b697: Status 404 returned error can't find the container with id 96f8c20e91d10accc968dc919e9971fb8ca255f33f8e41dab6da5e828b42b697 Jul 07 15:45:30.433071 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:30.431326 2585 generic.go:356] "Generic (PLEG): container finished" podID="6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3" containerID="fc0507642ce5f7639e3370902e612dc2ffcdfa5af590edfd488ed8b4b8605446" exitCode=0 Jul 07 15:45:30.433071 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:30.431435 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5bzj" event={"ID":"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3","Type":"ContainerDied","Data":"fc0507642ce5f7639e3370902e612dc2ffcdfa5af590edfd488ed8b4b8605446"} Jul 07 15:45:30.433071 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:30.432968 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"96f8c20e91d10accc968dc919e9971fb8ca255f33f8e41dab6da5e828b42b697"} Jul 07 15:45:31.439212 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.439152 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234" exitCode=0 Jul 07 15:45:31.439675 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.439267 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234"} Jul 07 15:45:31.442279 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.442252 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" event={"ID":"29156cfa-8705-442d-9ee7-32719a333b8c","Type":"ContainerStarted","Data":"b1086da9db5ac1fab836112934e8b9d46f102c477a96402233e305b7e621fe06"} Jul 07 15:45:31.444897 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.444871 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5bzj" event={"ID":"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3","Type":"ContainerStarted","Data":"20da34c04b46532b5b7bc2765c7d6e38bd2c5ec3b102c67f0a2b922576de46a3"} Jul 07 15:45:31.444897 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.444914 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5bzj" event={"ID":"6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3","Type":"ContainerStarted","Data":"d684ba8952ecc14d0f1d5ee281aa13d668ea6007d958cfb51f545168123fe2ca"} Jul 07 15:45:31.447523 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.447498 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" event={"ID":"59b448ea-38f8-4aec-aa8a-d379b07accff","Type":"ContainerStarted","Data":"7516bb0414f8329d10857ef7f91583a4cb5e5996d8cf44f5b6b4af3405e3f142"} Jul 07 15:45:31.447619 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.447553 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" event={"ID":"59b448ea-38f8-4aec-aa8a-d379b07accff","Type":"ContainerStarted","Data":"e7a7a7a9ce0d7961700e82d49fd9e3a8f52799b5feb2a7baccaf6efa7e855969"} Jul 07 15:45:31.447619 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.447580 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" event={"ID":"59b448ea-38f8-4aec-aa8a-d379b07accff","Type":"ContainerStarted","Data":"0211ae6f57128dcfcc4560f1574e609cf60bd69a9b6a0de4ac4f7a6c354cefea"} Jul 07 15:45:31.489857 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.489811 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nz7cp" podStartSLOduration=2.783603243 podStartE2EDuration="4.48979801s" totalImagesPullingTime="1.706194767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:31.488832206 +0000 UTC m=+65.980550670" watchObservedRunningTime="2026-07-07 15:45:31.48979801 +0000 UTC m=+65.981516473" Jul 07 15:45:31.510984 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.510944 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nspm8" podStartSLOduration=2.681461446 podStartE2EDuration="4.510933472s" totalImagesPullingTime="1.829472026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:31.510750863 +0000 UTC m=+66.002469327" watchObservedRunningTime="2026-07-07 15:45:31.510933472 +0000 UTC m=+66.002651961" Jul 07 15:45:31.530942 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.530904 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r5bzj" podStartSLOduration=3.411229521 podStartE2EDuration="4.530891288s" totalImagesPullingTime="1.03139218s" totalInitContainerRuntime="88.269587ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:31.530264418 +0000 UTC m=+66.021982882" watchObservedRunningTime="2026-07-07 15:45:31.530891288 +0000 UTC m=+66.022609750" Jul 07 15:45:31.817115 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.817085 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-549f79f6ff-bvd77"] Jul 07 15:45:31.819749 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.819712 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.822744 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.822718 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 15:45:31.823860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.823705 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 15:45:31.823860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.823724 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d5rxq\"" Jul 07 15:45:31.823860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.823743 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1brn94th184di\"" Jul 07 15:45:31.823860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.823807 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 15:45:31.823860 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.823843 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 15:45:31.826715 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.826693 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:31.826842 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.826759 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:31.829040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.829014 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:45:31.829133 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.829049 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:45:31.833403 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.833382 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-549f79f6ff-bvd77"] Jul 07 15:45:31.840270 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.840248 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/234a4877-fd36-4a8e-923e-24b3bee4b1bb-original-pull-secret\") pod \"global-pull-secret-syncer-dl6lx\" (UID: \"234a4877-fd36-4a8e-923e-24b3bee4b1bb\") " pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:31.840368 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.840283 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24014c1a-3f58-4032-9847-98ddf8a3cec0-metrics-certs\") pod \"network-metrics-daemon-wxh7c\" (UID: \"24014c1a-3f58-4032-9847-98ddf8a3cec0\") " pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:31.904374 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.904338 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb"] Jul 07 15:45:31.907018 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.906987 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:31.910851 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.910826 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 15:45:31.910971 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.910868 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rgdpr\"" Jul 07 15:45:31.917862 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.917831 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb"] Jul 07 15:45:31.927233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927213 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-client-ca-bundle\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927339 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927253 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb9fd8da-8f8c-47ec-bd66-f454929fa434-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927339 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927281 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb9fd8da-8f8c-47ec-bd66-f454929fa434-audit-log\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927445 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-secret-metrics-server-client-certs\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927484 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927439 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhcqk\" (UniqueName: \"kubernetes.io/projected/bb9fd8da-8f8c-47ec-bd66-f454929fa434-kube-api-access-jhcqk\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927484 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927479 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-secret-metrics-server-tls\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927556 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927498 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb9fd8da-8f8c-47ec-bd66-f454929fa434-metrics-server-audit-profiles\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:31.927605 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.927576 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:31.929931 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.929910 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:45:31.934144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.934128 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dl6lx" Jul 07 15:45:31.940479 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.940464 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:45:31.946614 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.946593 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wcvdn\"" Jul 07 15:45:31.951543 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.951521 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9l5b\" (UniqueName: \"kubernetes.io/projected/0c47ec45-4b0a-4a48-bf1e-08ba0838a96c-kube-api-access-n9l5b\") pod \"network-check-target-7dkvf\" (UID: \"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c\") " pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:31.955069 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:31.955052 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxh7c" Jul 07 15:45:32.027955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.027925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-secret-metrics-server-tls\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.027955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.027961 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb9fd8da-8f8c-47ec-bd66-f454929fa434-metrics-server-audit-profiles\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.028233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pczs2\" (UniqueName: \"kubernetes.io/projected/4933bd4b-56b2-4a4a-867a-4141f71cc75d-kube-api-access-pczs2\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.028233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028050 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-client-ca-bundle\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.028233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028065 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4933bd4b-56b2-4a4a-867a-4141f71cc75d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.028233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028117 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb9fd8da-8f8c-47ec-bd66-f454929fa434-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.028439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb9fd8da-8f8c-47ec-bd66-f454929fa434-audit-log\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.028439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028287 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-secret-metrics-server-client-certs\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.028439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028335 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhcqk\" (UniqueName: \"kubernetes.io/projected/bb9fd8da-8f8c-47ec-bd66-f454929fa434-kube-api-access-jhcqk\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.028999 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.028975 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb9fd8da-8f8c-47ec-bd66-f454929fa434-audit-log\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.029304 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.029141 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb9fd8da-8f8c-47ec-bd66-f454929fa434-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.029304 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.029269 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb9fd8da-8f8c-47ec-bd66-f454929fa434-metrics-server-audit-profiles\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.034612 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.031429 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-secret-metrics-server-client-certs\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.035447 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.035414 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-client-ca-bundle\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.035877 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.035857 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb9fd8da-8f8c-47ec-bd66-f454929fa434-secret-metrics-server-tls\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.037704 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.037663 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhcqk\" (UniqueName: \"kubernetes.io/projected/bb9fd8da-8f8c-47ec-bd66-f454929fa434-kube-api-access-jhcqk\") pod \"metrics-server-549f79f6ff-bvd77\" (UID: \"bb9fd8da-8f8c-47ec-bd66-f454929fa434\") " pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.129946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.129634 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:32.129946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.129774 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4933bd4b-56b2-4a4a-867a-4141f71cc75d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.129946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.129904 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pczs2\" (UniqueName: \"kubernetes.io/projected/4933bd4b-56b2-4a4a-867a-4141f71cc75d-kube-api-access-pczs2\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.130144 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:32.129961 2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 15:45:32.130144 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:45:32.130056 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4933bd4b-56b2-4a4a-867a-4141f71cc75d-monitoring-plugin-cert podName:4933bd4b-56b2-4a4a-867a-4141f71cc75d nodeName:}" failed. No retries permitted until 2026-07-07 15:45:32.630036349 +0000 UTC m=+67.121754795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4933bd4b-56b2-4a4a-867a-4141f71cc75d-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-mq9tb" (UID: "4933bd4b-56b2-4a4a-867a-4141f71cc75d") : secret "monitoring-plugin-cert" not found Jul 07 15:45:32.139486 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.139461 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pczs2\" (UniqueName: \"kubernetes.io/projected/4933bd4b-56b2-4a4a-867a-4141f71cc75d-kube-api-access-pczs2\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.243288 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.242951 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ctzps\"" Jul 07 15:45:32.250291 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.250230 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:32.306411 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.304880 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-549f79f6ff-bvd77"] Jul 07 15:45:32.310728 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:32.310678 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9fd8da_8f8c_47ec_bd66_f454929fa434.slice/crio-aa76618b70ce97c73971e257c842955b7ba097a1ecd909e99ab0ea59748fb2ad WatchSource:0}: Error finding container aa76618b70ce97c73971e257c842955b7ba097a1ecd909e99ab0ea59748fb2ad: Status 404 returned error can't find the container with id aa76618b70ce97c73971e257c842955b7ba097a1ecd909e99ab0ea59748fb2ad Jul 07 15:45:32.331193 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.329534 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxh7c"] Jul 07 15:45:32.342730 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:32.342705 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24014c1a_3f58_4032_9847_98ddf8a3cec0.slice/crio-61094ef1808b09e2d58985fdb58fdbfbcdc0ccb7665b91d91efbd5591fc09a35 WatchSource:0}: Error finding container 61094ef1808b09e2d58985fdb58fdbfbcdc0ccb7665b91d91efbd5591fc09a35: Status 404 returned error can't find the container with id 61094ef1808b09e2d58985fdb58fdbfbcdc0ccb7665b91d91efbd5591fc09a35 Jul 07 15:45:32.357821 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.357763 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dl6lx"] Jul 07 15:45:32.361515 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:32.361095 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234a4877_fd36_4a8e_923e_24b3bee4b1bb.slice/crio-d6a1b89b349c228e73d9f897c922f8ea933bf1b86dfd622cefe23fd65fddc6a4 WatchSource:0}: Error finding container d6a1b89b349c228e73d9f897c922f8ea933bf1b86dfd622cefe23fd65fddc6a4: Status 404 returned error can't find the container with id d6a1b89b349c228e73d9f897c922f8ea933bf1b86dfd622cefe23fd65fddc6a4 Jul 07 15:45:32.425786 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.425763 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7dkvf"] Jul 07 15:45:32.428890 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:32.428862 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c47ec45_4b0a_4a48_bf1e_08ba0838a96c.slice/crio-1f88f2c8d4e41311c6ff509a5e64807670e1e3683ba448d9e3a945730911259c WatchSource:0}: Error finding container 1f88f2c8d4e41311c6ff509a5e64807670e1e3683ba448d9e3a945730911259c: Status 404 returned error can't find the container with id 1f88f2c8d4e41311c6ff509a5e64807670e1e3683ba448d9e3a945730911259c Jul 07 15:45:32.455939 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.455870 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7dkvf" event={"ID":"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c","Type":"ContainerStarted","Data":"1f88f2c8d4e41311c6ff509a5e64807670e1e3683ba448d9e3a945730911259c"} Jul 07 15:45:32.457879 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.457462 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxh7c" event={"ID":"24014c1a-3f58-4032-9847-98ddf8a3cec0","Type":"ContainerStarted","Data":"61094ef1808b09e2d58985fdb58fdbfbcdc0ccb7665b91d91efbd5591fc09a35"} Jul 07 15:45:32.460430 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.460396 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"09f320d49e3db7e68d8020b3c07b34275af4a1ffc24f6d5fc535d0fefdd88ed1"} Jul 07 15:45:32.460533 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.460446 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"7e44c38ead0fd64ab0149f1ed578e22b29a934837cd835274fbb0b12ee382882"} Jul 07 15:45:32.460533 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.460474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"76592f4656a7994e5144b96e35d515d958e7c1e0429936981e1966fd9b9a51db"} Jul 07 15:45:32.462807 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.462759 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" event={"ID":"bb9fd8da-8f8c-47ec-bd66-f454929fa434","Type":"ContainerStarted","Data":"aa76618b70ce97c73971e257c842955b7ba097a1ecd909e99ab0ea59748fb2ad"} Jul 07 15:45:32.464487 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.464457 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dl6lx" event={"ID":"234a4877-fd36-4a8e-923e-24b3bee4b1bb","Type":"ContainerStarted","Data":"d6a1b89b349c228e73d9f897c922f8ea933bf1b86dfd622cefe23fd65fddc6a4"} Jul 07 15:45:32.497644 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.497610 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5ddd88968-p27t2"] Jul 07 15:45:32.500834 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.500812 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.503771 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.503743 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7qzn2\"" Jul 07 15:45:32.503991 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.503974 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 15:45:32.504089 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.504027 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 15:45:32.504089 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.504037 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 15:45:32.504374 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.504350 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 15:45:32.504469 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.504413 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 15:45:32.509603 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.509583 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 15:45:32.518099 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.518074 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5ddd88968-p27t2"] Jul 07 15:45:32.636285 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636211 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-telemeter-client-tls\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636285 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636258 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636285 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636279 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h26fz\" (UniqueName: \"kubernetes.io/projected/b51a8036-5a08-4fe1-9683-793eb3e26376-kube-api-access-h26fz\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636553 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636328 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-federate-client-tls\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636553 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636389 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-metrics-client-ca\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636553 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4933bd4b-56b2-4a4a-867a-4141f71cc75d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.636553 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636516 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-serving-certs-ca-bundle\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636744 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636562 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.636744 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.636602 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-secret-telemeter-client\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.639342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.639318 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4933bd4b-56b2-4a4a-867a-4141f71cc75d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-mq9tb\" (UID: \"4933bd4b-56b2-4a4a-867a-4141f71cc75d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:32.737591 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737555 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-serving-certs-ca-bundle\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.737817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737612 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.737817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737646 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-secret-telemeter-client\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.737817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737680 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-telemeter-client-tls\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.737817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737721 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.737817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h26fz\" (UniqueName: \"kubernetes.io/projected/b51a8036-5a08-4fe1-9683-793eb3e26376-kube-api-access-h26fz\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.737817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-federate-client-tls\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.738120 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.737839 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-metrics-client-ca\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.738416 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.738392 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-serving-certs-ca-bundle\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.738676 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.738511 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-metrics-client-ca\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.739075 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.739015 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51a8036-5a08-4fe1-9683-793eb3e26376-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.740890 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.740861 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.741427 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.741404 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-telemeter-client-tls\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.741427 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.741417 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-federate-client-tls\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.741821 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.741800 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b51a8036-5a08-4fe1-9683-793eb3e26376-secret-telemeter-client\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.747541 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.747514 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h26fz\" (UniqueName: \"kubernetes.io/projected/b51a8036-5a08-4fe1-9683-793eb3e26376-kube-api-access-h26fz\") pod \"telemeter-client-5ddd88968-p27t2\" (UID: \"b51a8036-5a08-4fe1-9683-793eb3e26376\") " pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.812475 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.812435 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" Jul 07 15:45:32.818521 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:32.818493 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:33.022204 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.021518 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5ddd88968-p27t2"] Jul 07 15:45:33.046189 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.046150 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb"] Jul 07 15:45:33.048955 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:45:33.048901 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4933bd4b_56b2_4a4a_867a_4141f71cc75d.slice/crio-3e78bc64b3972cb91c5db637a91cd36f3f438e85351ed69e749b598a4481d6d8 WatchSource:0}: Error finding container 3e78bc64b3972cb91c5db637a91cd36f3f438e85351ed69e749b598a4481d6d8: Status 404 returned error can't find the container with id 3e78bc64b3972cb91c5db637a91cd36f3f438e85351ed69e749b598a4481d6d8 Jul 07 15:45:33.472550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.472461 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" event={"ID":"4933bd4b-56b2-4a4a-867a-4141f71cc75d","Type":"ContainerStarted","Data":"3e78bc64b3972cb91c5db637a91cd36f3f438e85351ed69e749b598a4481d6d8"} Jul 07 15:45:33.476107 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.476076 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" event={"ID":"b51a8036-5a08-4fe1-9683-793eb3e26376","Type":"ContainerStarted","Data":"7d7382ad2faa3c138f804fa7563b07fc49b230389ff2095dff2fa32c41a4cd9c"} Jul 07 15:45:33.646797 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.646760 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:33.685252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.683668 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:33.685252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.683859 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.687539 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.687515 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4cm8hukvsm84v\"" Jul 07 15:45:33.687783 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.687761 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 15:45:33.688014 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.687996 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 15:45:33.688299 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.688283 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9hbv4\"" Jul 07 15:45:33.688432 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.688416 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 15:45:33.688516 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.688290 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 15:45:33.688516 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.688366 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 15:45:33.689341 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.689312 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 15:45:33.690045 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.690024 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 15:45:33.690264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.690247 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 15:45:33.692412 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.690989 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 15:45:33.692412 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.691225 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 15:45:33.692412 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.691455 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 15:45:33.692412 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.692030 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 15:45:33.693387 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.693368 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 15:45:33.698600 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.698577 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749650 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749701 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749729 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749766 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-config\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749788 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749810 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749834 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749865 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749906 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749931 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749958 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.749997 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750025 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750058 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750107 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750180 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750237 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2swjg\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-kube-api-access-2swjg\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.751930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.750273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850753 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850804 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850847 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850883 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850907 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850929 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850959 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.850983 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851010 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851050 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851075 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851119 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851136 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2swjg\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-kube-api-access-2swjg\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851175 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851226 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.851439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.852253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851281 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.852253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851315 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-config\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.852253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.851337 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.854227 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.853577 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.854227 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.853623 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.854402 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.854225 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.854402 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.854359 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.856301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.855616 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.856301 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.855766 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.857621 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.857505 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.858848 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.858782 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.860014 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.859963 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.860134 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.860112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.860895 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.860853 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.861868 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.861846 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.865208 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.865043 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.865325 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.865258 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.865803 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.865724 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.865897 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.865801 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.866057 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.866029 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-config\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.866922 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.866897 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2swjg\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-kube-api-access-2swjg\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.867122 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.867100 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:33.871692 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.871614 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:34.000209 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:33.999882 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:36.410607 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:36.410572 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vnw9p" Jul 07 15:45:38.997432 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:38.997328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:39.508790 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.508749 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"81fe0843d1c603f9e6ddd6851bf5e9f5d6c3973c465f9a35e3b245a7dae8f3dd"} Jul 07 15:45:39.508942 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.508868 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"49adffa1a193dc345176bf1124718f857ad46ccae1ea55a9441ad4c730d6cad6"} Jul 07 15:45:39.508942 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.508914 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" event={"ID":"4e6258fa-5b77-4809-a472-fe0f23960f80","Type":"ContainerStarted","Data":"be791f376ff8c95ff44cff94bf5392b6682e969db0f3218d63864c065db63fcc"} Jul 07 15:45:39.509074 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.508960 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:39.510493 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.510465 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" event={"ID":"bb9fd8da-8f8c-47ec-bd66-f454929fa434","Type":"ContainerStarted","Data":"ead2703a55584434fe717aa00a99630e89796e9a6ac8f54e26cfc8be73e9ebc3"} Jul 07 15:45:39.511925 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.511894 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dl6lx" event={"ID":"234a4877-fd36-4a8e-923e-24b3bee4b1bb","Type":"ContainerStarted","Data":"6e64273299498eca28a53515f3bb5a6767543c1ced069f338122e59dd698a655"} Jul 07 15:45:39.513916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.513891 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" event={"ID":"4933bd4b-56b2-4a4a-867a-4141f71cc75d","Type":"ContainerStarted","Data":"12db3b09cffad626291d34304799eddf6d0e0a58abde227e24d0a901e687969c"} Jul 07 15:45:39.514037 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.513945 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:39.515236 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.515217 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" Jul 07 15:45:39.515712 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.515692 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" exitCode=0 Jul 07 15:45:39.515795 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.515782 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} Jul 07 15:45:39.515861 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.515842 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"85202e844a52bfd3a9a9da1fdce3eccd4d13628cab50b683413197552fac6265"} Jul 07 15:45:39.518970 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.518948 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" Jul 07 15:45:39.519645 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.519624 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c"} Jul 07 15:45:39.519745 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.519687 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c"} Jul 07 15:45:39.519745 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.519731 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61"} Jul 07 15:45:39.519853 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.519769 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618"} Jul 07 15:45:39.519853 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.519812 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5"} Jul 07 15:45:39.519937 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.519853 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerStarted","Data":"3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9"} Jul 07 15:45:39.521105 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.521082 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7dkvf" event={"ID":"0c47ec45-4b0a-4a48-bf1e-08ba0838a96c","Type":"ContainerStarted","Data":"a8b3c6010f54b7a76f301f498904524f0dbce93e75510232bb95a2f741550c2c"} Jul 07 15:45:39.521233 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.521215 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:45:39.523291 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.523272 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxh7c" event={"ID":"24014c1a-3f58-4032-9847-98ddf8a3cec0","Type":"ContainerStarted","Data":"9862b8f3d24ae73e15446e629d785a932f19cad808033967bb5e258ff922eaa9"} Jul 07 15:45:39.523369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.523306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxh7c" event={"ID":"24014c1a-3f58-4032-9847-98ddf8a3cec0","Type":"ContainerStarted","Data":"74ba3b7ab6034ad473014a4b096cc693c787a4ca86c813aa95919bed8be9354f"} Jul 07 15:45:39.525198 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.525181 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" event={"ID":"b51a8036-5a08-4fe1-9683-793eb3e26376","Type":"ContainerStarted","Data":"4f480b9b4718fecd08fb0a5abcf715f486bcfe243dcd3c857b305df5b985f2a2"} Jul 07 15:45:39.525274 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.525217 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" event={"ID":"b51a8036-5a08-4fe1-9683-793eb3e26376","Type":"ContainerStarted","Data":"8b8f80a439cb17e12fceeec453aedd7dd0a6c1903d0f841a997b0eb0623cecb8"} Jul 07 15:45:39.525274 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.525235 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" event={"ID":"b51a8036-5a08-4fe1-9683-793eb3e26376","Type":"ContainerStarted","Data":"f95a3e9469485b39a5b54f76f8057c000e37c28b613ec8d91e07bba58a72d496"} Jul 07 15:45:39.531744 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.531693 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d68f8b574-62sm9" podStartSLOduration=2.016861451 podStartE2EDuration="10.531680491s" totalImagesPullingTime="8.51481904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.5306494 +0000 UTC m=+74.022367863" watchObservedRunningTime="2026-07-07 15:45:39.531680491 +0000 UTC m=+74.023398968" Jul 07 15:45:39.576361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.576308 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" podStartSLOduration=2.040369125 podStartE2EDuration="8.576291879s" totalImagesPullingTime="6.535922754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.575258309 +0000 UTC m=+74.066976777" watchObservedRunningTime="2026-07-07 15:45:39.576291879 +0000 UTC m=+74.068010342" Jul 07 15:45:39.597938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.597656 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-mq9tb" podStartSLOduration=2.797446168 podStartE2EDuration="8.597639268s" totalImagesPullingTime="5.8001931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.596274334 +0000 UTC m=+74.087992809" watchObservedRunningTime="2026-07-07 15:45:39.597639268 +0000 UTC m=+74.089357735" Jul 07 15:45:39.615480 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.615436 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dl6lx" podStartSLOduration=67.096651037 podStartE2EDuration="1m13.615420622s" totalImagesPullingTime="6.518769585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.613715016 +0000 UTC m=+74.105433479" watchObservedRunningTime="2026-07-07 15:45:39.615420622 +0000 UTC m=+74.107139086" Jul 07 15:45:39.634003 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.633957 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wxh7c" podStartSLOduration=67.13635654 podStartE2EDuration="1m13.63394342s" totalImagesPullingTime="6.49758688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.632719672 +0000 UTC m=+74.124438146" watchObservedRunningTime="2026-07-07 15:45:39.63394342 +0000 UTC m=+74.125661906" Jul 07 15:45:39.679714 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.679658 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.945880687 podStartE2EDuration="11.679637211s" totalImagesPullingTime="8.70294113s" totalInitContainerRuntime="30.815394ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.677866331 +0000 UTC m=+74.169584793" watchObservedRunningTime="2026-07-07 15:45:39.679637211 +0000 UTC m=+74.171355678" Jul 07 15:45:39.750057 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.750009 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5ddd88968-p27t2" podStartSLOduration=1.9140637 podStartE2EDuration="7.749993979s" totalImagesPullingTime="5.835930279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.748961467 +0000 UTC m=+74.240679932" watchObservedRunningTime="2026-07-07 15:45:39.749993979 +0000 UTC m=+74.241712462" Jul 07 15:45:39.804092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:39.804038 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7dkvf" podStartSLOduration=67.386139775 podStartE2EDuration="1m13.804020933s" totalImagesPullingTime="6.417881158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:39.802981684 +0000 UTC m=+74.294700148" watchObservedRunningTime="2026-07-07 15:45:39.804020933 +0000 UTC m=+74.295739396" Jul 07 15:45:43.545729 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.545689 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} Jul 07 15:45:43.546104 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.545754 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} Jul 07 15:45:43.546104 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.545784 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} Jul 07 15:45:43.546104 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.545843 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} Jul 07 15:45:43.546104 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.545866 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} Jul 07 15:45:43.546104 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.545888 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerStarted","Data":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} Jul 07 15:45:43.578658 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:43.578603 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.278580112 podStartE2EDuration="10.5785862s" totalImagesPullingTime="3.236811365s" totalInitContainerRuntime="63.194723ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:43.576148369 +0000 UTC m=+78.067866865" watchObservedRunningTime="2026-07-07 15:45:43.5785862 +0000 UTC m=+78.070304662" Jul 07 15:45:44.000425 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:44.000390 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:52.130217 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:52.130063 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:45:52.130217 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:45:52.130140 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:46:10.533229 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:10.533200 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7dkvf" Jul 07 15:46:12.135896 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:12.135868 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:46:12.139626 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:12.139603 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-549f79f6ff-bvd77" Jul 07 15:46:34.000791 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:34.000747 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:34.021388 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:34.021363 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:34.766380 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:34.766356 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:47.704092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704054 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:46:47.704688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704386 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="alertmanager" containerID="cri-o://3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9" gracePeriod=120 Jul 07 15:46:47.704688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704437 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-metric" containerID="cri-o://3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c" gracePeriod=120 Jul 07 15:46:47.704688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704481 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="config-reloader" containerID="cri-o://4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5" gracePeriod=120 Jul 07 15:46:47.704688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704481 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-web" containerID="cri-o://a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618" gracePeriod=120 Jul 07 15:46:47.704688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704532 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="prom-label-proxy" containerID="cri-o://cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c" gracePeriod=120 Jul 07 15:46:47.704688 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:47.704621 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy" containerID="cri-o://28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61" gracePeriod=120 Jul 07 15:46:48.806495 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806465 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c" exitCode=0 Jul 07 15:46:48.806495 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806491 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c" exitCode=0 Jul 07 15:46:48.806495 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806499 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61" exitCode=0 Jul 07 15:46:48.806495 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806505 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5" exitCode=0 Jul 07 15:46:48.806938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806509 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9" exitCode=0 Jul 07 15:46:48.806938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806535 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c"} Jul 07 15:46:48.806938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806601 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c"} Jul 07 15:46:48.806938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806628 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61"} Jul 07 15:46:48.806938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806661 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5"} Jul 07 15:46:48.806938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.806685 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9"} Jul 07 15:46:48.955204 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:48.955181 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.009711 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.009683 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.009711 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.009714 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-metric\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.009949 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.009743 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-config-out\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.009949 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.009763 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-web\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.009949 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.009785 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-main-db\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.010126 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010097 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:46:49.010278 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010232 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-trusted-ca-bundle\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.010278 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010272 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-web-config\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.010395 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010310 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-config-volume\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010565 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27289\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-kube-api-access-27289\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010636 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-cluster-tls-config\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010641 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010664 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-tls-assets\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010709 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.010757 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-metrics-client-ca\") pod \"855d73b5-0aec-4fca-929b-9ba10bb89881\" (UID: \"855d73b5-0aec-4fca-929b-9ba10bb89881\") " Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.011065 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-main-db\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.011087 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.011546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.011501 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:49.015362 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.015316 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.015662 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.015599 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-config-out" (OuterVolumeSpecName: "config-out") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:46:49.015951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.015904 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:49.016044 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.015972 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.016116 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.016092 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.016287 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.016265 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-config-volume" (OuterVolumeSpecName: "config-volume") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.016439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.016408 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.016705 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.016683 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-kube-api-access-27289" (OuterVolumeSpecName: "kube-api-access-27289") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "kube-api-access-27289". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:49.020366 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.020339 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.028434 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.028410 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-web-config" (OuterVolumeSpecName: "web-config") pod "855d73b5-0aec-4fca-929b-9ba10bb89881" (UID: "855d73b5-0aec-4fca-929b-9ba10bb89881"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:49.111894 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111857 2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-cluster-tls-config\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.111894 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111888 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-tls-assets\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.111894 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111898 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-main-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.111894 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111906 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855d73b5-0aec-4fca-929b-9ba10bb89881-metrics-client-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111915 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111927 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111936 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855d73b5-0aec-4fca-929b-9ba10bb89881-config-out\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111945 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111953 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-web-config\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111961 2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855d73b5-0aec-4fca-929b-9ba10bb89881-config-volume\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.112187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.111968 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27289\" (UniqueName: \"kubernetes.io/projected/855d73b5-0aec-4fca-929b-9ba10bb89881-kube-api-access-27289\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:49.813553 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.813522 2585 generic.go:356] "Generic (PLEG): container finished" podID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerID="a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618" exitCode=0 Jul 07 15:46:49.813955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.813598 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618"} Jul 07 15:46:49.813955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.813664 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855d73b5-0aec-4fca-929b-9ba10bb89881","Type":"ContainerDied","Data":"9b40dccf30dbcad17017ffb9e19756943254a4359efd8362f005895b77ddb760"} Jul 07 15:46:49.813955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.813608 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.813955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.813696 2585 scope.go:117] "RemoveContainer" containerID="cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c" Jul 07 15:46:49.822516 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.822418 2585 scope.go:117] "RemoveContainer" containerID="3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c" Jul 07 15:46:49.829603 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.829587 2585 scope.go:117] "RemoveContainer" containerID="28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61" Jul 07 15:46:49.836760 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.836699 2585 scope.go:117] "RemoveContainer" containerID="a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618" Jul 07 15:46:49.839372 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.839350 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:46:49.845656 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.845552 2585 scope.go:117] "RemoveContainer" containerID="4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5" Jul 07 15:46:49.848120 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.848095 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:46:49.853845 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.853827 2585 scope.go:117] "RemoveContainer" containerID="3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9" Jul 07 15:46:49.861633 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.861606 2585 scope.go:117] "RemoveContainer" containerID="793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234" Jul 07 15:46:49.869662 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.869641 2585 scope.go:117] "RemoveContainer" containerID="cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c" Jul 07 15:46:49.869960 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.869943 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c\": container with ID starting with cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c not found: ID does not exist" containerID="cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c" Jul 07 15:46:49.870045 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.869969 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c"} err="failed to get container status \"cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c\": rpc error: code = NotFound desc = could not find container \"cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c\": container with ID starting with cf4f56e0df4bdc7cb9e9e9f9148ef4f2325ed5f13d0ec7dc0160fbc1d29d919c not found: ID does not exist" Jul 07 15:46:49.870045 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870004 2585 scope.go:117] "RemoveContainer" containerID="3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c" Jul 07 15:46:49.870302 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.870285 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c\": container with ID starting with 3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c not found: ID does not exist" containerID="3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c" Jul 07 15:46:49.870369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870305 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c"} err="failed to get container status \"3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c\": rpc error: code = NotFound desc = could not find container \"3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c\": container with ID starting with 3880556cfb4c22f91ed3bc0ebb3d611dcac6ddaf6de7a6dc9422f2906d23695c not found: ID does not exist" Jul 07 15:46:49.870369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870320 2585 scope.go:117] "RemoveContainer" containerID="28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61" Jul 07 15:46:49.870511 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.870495 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61\": container with ID starting with 28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61 not found: ID does not exist" containerID="28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61" Jul 07 15:46:49.870546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870514 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61"} err="failed to get container status \"28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61\": rpc error: code = NotFound desc = could not find container \"28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61\": container with ID starting with 28221571ac076e7ce14cf1949f0b356d638f41ce3904fa5a3be14acad25b0e61 not found: ID does not exist" Jul 07 15:46:49.870546 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870526 2585 scope.go:117] "RemoveContainer" containerID="a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618" Jul 07 15:46:49.870754 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.870739 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618\": container with ID starting with a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618 not found: ID does not exist" containerID="a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618" Jul 07 15:46:49.870793 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870756 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618"} err="failed to get container status \"a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618\": rpc error: code = NotFound desc = could not find container \"a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618\": container with ID starting with a8f36e9e040dac9968dd323f0062795e1da722285f92520f6b8b173c84e3e618 not found: ID does not exist" Jul 07 15:46:49.870793 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870767 2585 scope.go:117] "RemoveContainer" containerID="4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5" Jul 07 15:46:49.870992 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.870976 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5\": container with ID starting with 4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5 not found: ID does not exist" containerID="4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5" Jul 07 15:46:49.871039 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.870995 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5"} err="failed to get container status \"4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5\": rpc error: code = NotFound desc = could not find container \"4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5\": container with ID starting with 4356c6d9aacf6d0b88f2f48028beda90debb24c54113fd30ca9b454d942a60e5 not found: ID does not exist" Jul 07 15:46:49.871039 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.871007 2585 scope.go:117] "RemoveContainer" containerID="3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9" Jul 07 15:46:49.871211 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.871197 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9\": container with ID starting with 3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9 not found: ID does not exist" containerID="3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9" Jul 07 15:46:49.871256 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.871214 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9"} err="failed to get container status \"3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9\": rpc error: code = NotFound desc = could not find container \"3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9\": container with ID starting with 3335d4555bb714380e46a3314b334794bb96154f6243968017876dc6e11dc4f9 not found: ID does not exist" Jul 07 15:46:49.871256 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.871232 2585 scope.go:117] "RemoveContainer" containerID="793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234" Jul 07 15:46:49.871428 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:49.871414 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234\": container with ID starting with 793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234 not found: ID does not exist" containerID="793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234" Jul 07 15:46:49.871466 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.871429 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234"} err="failed to get container status \"793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234\": rpc error: code = NotFound desc = could not find container \"793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234\": container with ID starting with 793c2bd61e35f5ddf6e6c470d140aa39b8324708fe2db69e8c07ea3ef831f234 not found: ID does not exist" Jul 07 15:46:49.876679 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876659 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:46:49.876951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876939 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy" Jul 07 15:46:49.876990 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876953 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy" Jul 07 15:46:49.876990 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876971 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-metric" Jul 07 15:46:49.876990 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876976 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-metric" Jul 07 15:46:49.876990 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876985 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-web" Jul 07 15:46:49.876990 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876991 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-web" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.876997 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="init-config-reloader" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877002 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="init-config-reloader" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877013 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="prom-label-proxy" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877019 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="prom-label-proxy" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877026 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="config-reloader" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877031 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="config-reloader" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877036 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="alertmanager" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877041 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="alertmanager" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877083 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877090 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-metric" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877096 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="kube-rbac-proxy-web" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877103 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="config-reloader" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877108 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="alertmanager" Jul 07 15:46:49.877131 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.877113 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" containerName="prom-label-proxy" Jul 07 15:46:49.880539 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.880524 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.883597 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.883570 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 15:46:49.883708 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.883632 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 15:46:49.884610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.883836 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 15:46:49.884610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.884056 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 15:46:49.884610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.884203 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zkfx4\"" Jul 07 15:46:49.884610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.884406 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 15:46:49.884610 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.884437 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 15:46:49.885285 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.885133 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 15:46:49.885444 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.885425 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 15:46:49.888506 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.888487 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 15:46:49.896585 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.896560 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:46:49.918456 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918416 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918654 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918472 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143bf90f-71c5-4af3-aeda-f7440be81da0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918654 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918529 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918654 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918559 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918654 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918591 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kss2d\" (UniqueName: \"kubernetes.io/projected/143bf90f-71c5-4af3-aeda-f7440be81da0-kube-api-access-kss2d\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918654 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918644 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143bf90f-71c5-4af3-aeda-f7440be81da0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918687 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918715 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918744 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-config-volume\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918796 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143bf90f-71c5-4af3-aeda-f7440be81da0-config-out\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.918918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/143bf90f-71c5-4af3-aeda-f7440be81da0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.919153 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918908 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143bf90f-71c5-4af3-aeda-f7440be81da0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:49.919153 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:49.918948 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-web-config\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019699 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019664 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019699 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019708 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143bf90f-71c5-4af3-aeda-f7440be81da0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019774 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kss2d\" (UniqueName: \"kubernetes.io/projected/143bf90f-71c5-4af3-aeda-f7440be81da0-kube-api-access-kss2d\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143bf90f-71c5-4af3-aeda-f7440be81da0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019871 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-config-volume\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019927 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143bf90f-71c5-4af3-aeda-f7440be81da0-config-out\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.019964 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019951 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/143bf90f-71c5-4af3-aeda-f7440be81da0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.020443 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.019979 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143bf90f-71c5-4af3-aeda-f7440be81da0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.020443 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.020003 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-web-config\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.021723 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.021659 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143bf90f-71c5-4af3-aeda-f7440be81da0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023143 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023118 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023261 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023202 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023331 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023259 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023391 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023364 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023477 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023460 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023539 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023459 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/143bf90f-71c5-4af3-aeda-f7440be81da0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.023842 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.023822 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143bf90f-71c5-4af3-aeda-f7440be81da0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.024032 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.024014 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-web-config\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.024133 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.024112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143bf90f-71c5-4af3-aeda-f7440be81da0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.024225 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.024135 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/143bf90f-71c5-4af3-aeda-f7440be81da0-config-volume\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.025110 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.025092 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143bf90f-71c5-4af3-aeda-f7440be81da0-config-out\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.028612 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.028588 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kss2d\" (UniqueName: \"kubernetes.io/projected/143bf90f-71c5-4af3-aeda-f7440be81da0-kube-api-access-kss2d\") pod \"alertmanager-main-0\" (UID: \"143bf90f-71c5-4af3-aeda-f7440be81da0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.037272 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.037245 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855d73b5-0aec-4fca-929b-9ba10bb89881" path="/var/lib/kubelet/pods/855d73b5-0aec-4fca-929b-9ba10bb89881/volumes" Jul 07 15:46:50.191951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.191909 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:46:50.327825 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.327800 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:46:50.330127 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:46:50.330104 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143bf90f_71c5_4af3_aeda_f7440be81da0.slice/crio-10d55262908c5030ab01816305fcbdbe226059223a66b8d80f2047e2ec251e28 WatchSource:0}: Error finding container 10d55262908c5030ab01816305fcbdbe226059223a66b8d80f2047e2ec251e28: Status 404 returned error can't find the container with id 10d55262908c5030ab01816305fcbdbe226059223a66b8d80f2047e2ec251e28 Jul 07 15:46:50.819097 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.819065 2585 generic.go:356] "Generic (PLEG): container finished" podID="143bf90f-71c5-4af3-aeda-f7440be81da0" containerID="80e4373a89bed5fddda894fd34219141723b484933609ea8d48c43e225c74030" exitCode=0 Jul 07 15:46:50.819569 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.819174 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerDied","Data":"80e4373a89bed5fddda894fd34219141723b484933609ea8d48c43e225c74030"} Jul 07 15:46:50.819569 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:50.819262 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"10d55262908c5030ab01816305fcbdbe226059223a66b8d80f2047e2ec251e28"} Jul 07 15:46:51.832618 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.832578 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"11a70299b4e9bdf30996a26819b469038eeadc61b66fbe7a9312c1315913787d"} Jul 07 15:46:51.833072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.832673 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"848de1b394f0d28456bb4e0839345d14f55f2c737dae47966b490fda8775671c"} Jul 07 15:46:51.833072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.832702 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"d65a80c122bf4f9e4c8d12fc4ccf270e1144c0f3a1ffabee736e342e10f70721"} Jul 07 15:46:51.833072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.832725 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"50ca6ff8d15abd64466cf0f431311813728c3400f34e50abd7c0dc241eee1779"} Jul 07 15:46:51.833072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.832748 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"7511a8089e8226410776b3498f99a34ac9d510fc309805336a104851ee757cb0"} Jul 07 15:46:51.833072 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.832772 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"143bf90f-71c5-4af3-aeda-f7440be81da0","Type":"ContainerStarted","Data":"cd2aecbd4c891a767635c7af443c2c041c1c6a78b4ffe1af4cd681357e4e9471"} Jul 07 15:46:51.864633 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.864568 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8493974250000003 podStartE2EDuration="2.864545337s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.147912ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:51.864055859 +0000 UTC m=+146.355774324" watchObservedRunningTime="2026-07-07 15:46:51.864545337 +0000 UTC m=+146.356263801" Jul 07 15:46:51.971910 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.971875 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:46:51.972364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.972184 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="prometheus" containerID="cri-o://2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" gracePeriod=600 Jul 07 15:46:51.972364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.972215 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy" containerID="cri-o://c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" gracePeriod=600 Jul 07 15:46:51.972364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.972248 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-web" containerID="cri-o://a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" gracePeriod=600 Jul 07 15:46:51.972364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.972276 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-thanos" containerID="cri-o://53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" gracePeriod=600 Jul 07 15:46:51.972364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.972270 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="config-reloader" containerID="cri-o://2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" gracePeriod=600 Jul 07 15:46:51.972364 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:51.972246 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="thanos-sidecar" containerID="cri-o://a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" gracePeriod=600 Jul 07 15:46:52.244137 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.244103 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.341830 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-metrics-client-ca\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.341914 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-web-config\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.341942 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-kubelet-serving-ca-bundle\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.341976 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2swjg\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-kube-api-access-2swjg\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342031 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342059 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-1\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342089 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-db\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342128 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-thanos-prometheus-http-client-file\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342189 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-2\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342217 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-config\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342257 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-kube-rbac-proxy\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342287 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-grpc-tls\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342317 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-tls\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342358 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-serving-certs-ca-bundle\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342386 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-tls-assets\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342437 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-config-out\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342473 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-metrics-client-certs\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344066 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342502 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-0\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344974 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342538 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344974 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.342570 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-trusted-ca-bundle\") pod \"eaf41a1d-72e5-4f33-9341-8426efeec099\" (UID: \"eaf41a1d-72e5-4f33-9341-8426efeec099\") " Jul 07 15:46:52.344974 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.343342 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.344974 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.343729 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.347570 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.347147 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.347570 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.347550 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.348462 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.348382 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.349135 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.348713 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.349297 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.349259 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:52.349374 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.349341 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:46:52.351414 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.351375 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:52.351497 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.351411 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.351566 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.351502 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.351686 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.351639 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.352182 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.352139 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-config-out" (OuterVolumeSpecName: "config-out") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:46:52.352276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.352220 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.352442 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.352407 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.352651 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.352615 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.352781 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.352759 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-kube-api-access-2swjg" (OuterVolumeSpecName: "kube-api-access-2swjg") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "kube-api-access-2swjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:52.352894 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.352871 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-config" (OuterVolumeSpecName: "config") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.353046 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.353025 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.360642 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.360620 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-web-config" (OuterVolumeSpecName: "web-config") pod "eaf41a1d-72e5-4f33-9341-8426efeec099" (UID: "eaf41a1d-72e5-4f33-9341-8426efeec099"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:52.443838 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443798 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.443838 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443829 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-db\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.443838 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443840 2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.443838 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443849 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443859 2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-config\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443868 2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-kube-rbac-proxy\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443876 2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-grpc-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443885 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443892 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443902 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-tls-assets\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443910 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaf41a1d-72e5-4f33-9341-8426efeec099-config-out\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443917 2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-metrics-client-certs\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443936 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443945 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443954 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443962 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-metrics-client-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443970 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-web-config\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443978 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaf41a1d-72e5-4f33-9341-8426efeec099-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443986 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2swjg\" (UniqueName: \"kubernetes.io/projected/eaf41a1d-72e5-4f33-9341-8426efeec099-kube-api-access-2swjg\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.444087 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.443994 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaf41a1d-72e5-4f33-9341-8426efeec099-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:46:52.840332 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840302 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" exitCode=0 Jul 07 15:46:52.840332 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840328 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" exitCode=0 Jul 07 15:46:52.840332 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840336 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" exitCode=0 Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840343 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" exitCode=0 Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840351 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" exitCode=0 Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840357 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" exitCode=0 Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840382 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840467 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840503 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840548 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840589 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840574 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840630 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840671 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} Jul 07 15:46:52.840809 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.840712 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaf41a1d-72e5-4f33-9341-8426efeec099","Type":"ContainerDied","Data":"85202e844a52bfd3a9a9da1fdce3eccd4d13628cab50b683413197552fac6265"} Jul 07 15:46:52.848909 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.848895 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.856017 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.856001 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.862710 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.862683 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.865326 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.865305 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:46:52.870245 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.870229 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.870813 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.870793 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:46:52.877008 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.876991 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.884057 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.884042 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.890643 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.890629 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.890883 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.890862 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.890943 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.890890 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} err="failed to get container status \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" Jul 07 15:46:52.890943 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.890909 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.891199 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.891154 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.891283 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891208 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} err="failed to get container status \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" Jul 07 15:46:52.891283 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891229 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.891481 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.891466 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.891528 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891482 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} err="failed to get container status \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" Jul 07 15:46:52.891528 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891493 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.891708 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.891687 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.891750 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891714 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} err="failed to get container status \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" Jul 07 15:46:52.891750 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891728 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.891951 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.891937 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.891988 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891954 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} err="failed to get container status \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" Jul 07 15:46:52.891988 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.891967 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.892227 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.892206 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.892262 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892235 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} err="failed to get container status \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" Jul 07 15:46:52.892262 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892256 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.892472 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:46:52.892457 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.892507 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892475 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} err="failed to get container status \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" Jul 07 15:46:52.892507 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892488 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.892703 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892687 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} err="failed to get container status \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" Jul 07 15:46:52.892703 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892702 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.892922 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892906 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} err="failed to get container status \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" Jul 07 15:46:52.892922 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.892922 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.893133 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893118 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} err="failed to get container status \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" Jul 07 15:46:52.893195 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893135 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.893361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893346 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} err="failed to get container status \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" Jul 07 15:46:52.893361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893360 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.893586 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893570 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} err="failed to get container status \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" Jul 07 15:46:52.893622 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893586 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.893792 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893775 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} err="failed to get container status \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" Jul 07 15:46:52.893832 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893792 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.894009 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.893991 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} err="failed to get container status \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" Jul 07 15:46:52.894062 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894009 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.894264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894247 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} err="failed to get container status \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" Jul 07 15:46:52.894309 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894264 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.894485 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894469 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} err="failed to get container status \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" Jul 07 15:46:52.894526 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894485 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.894700 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894682 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} err="failed to get container status \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" Jul 07 15:46:52.894770 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894713 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.894958 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894932 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} err="failed to get container status \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" Jul 07 15:46:52.895017 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.894959 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.895204 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895187 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} err="failed to get container status \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" Jul 07 15:46:52.895262 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895204 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.895412 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895393 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} err="failed to get container status \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" Jul 07 15:46:52.895482 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895414 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.895628 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895611 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} err="failed to get container status \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" Jul 07 15:46:52.895666 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895628 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.895811 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895795 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} err="failed to get container status \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" Jul 07 15:46:52.895866 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895812 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.896003 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.895988 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} err="failed to get container status \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" Jul 07 15:46:52.896052 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896003 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.896223 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896204 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} err="failed to get container status \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" Jul 07 15:46:52.896277 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896224 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.896425 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896408 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} err="failed to get container status \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" Jul 07 15:46:52.896467 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896425 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.896625 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896606 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} err="failed to get container status \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" Jul 07 15:46:52.896669 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896626 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.896803 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896787 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} err="failed to get container status \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" Jul 07 15:46:52.896844 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.896803 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.897040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897023 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} err="failed to get container status \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" Jul 07 15:46:52.897108 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897042 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.897380 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897364 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} err="failed to get container status \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" Jul 07 15:46:52.897439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897380 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.897580 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897557 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} err="failed to get container status \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" Jul 07 15:46:52.897626 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897582 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.897790 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897770 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} err="failed to get container status \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" Jul 07 15:46:52.897790 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897793 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.898008 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.897987 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} err="failed to get container status \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" Jul 07 15:46:52.898008 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898010 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.898415 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898379 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} err="failed to get container status \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" Jul 07 15:46:52.898415 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898403 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.898711 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898690 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} err="failed to get container status \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" Jul 07 15:46:52.898711 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898712 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.898978 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898948 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} err="failed to get container status \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" Jul 07 15:46:52.899033 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.898981 2585 scope.go:117] "RemoveContainer" containerID="53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e" Jul 07 15:46:52.899268 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.899247 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e"} err="failed to get container status \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": rpc error: code = NotFound desc = could not find container \"53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e\": container with ID starting with 53f327c6f1d041de0ebc9de6bdac701a540e578c33c2dd103881d0fd5eaa735e not found: ID does not exist" Jul 07 15:46:52.899344 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.899267 2585 scope.go:117] "RemoveContainer" containerID="c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952" Jul 07 15:46:52.899526 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.899499 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952"} err="failed to get container status \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": rpc error: code = NotFound desc = could not find container \"c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952\": container with ID starting with c4023b34e0d4e0db1cb64bc54514c6f1d05d59387e73fe6d8206e66755364952 not found: ID does not exist" Jul 07 15:46:52.899526 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.899527 2585 scope.go:117] "RemoveContainer" containerID="a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb" Jul 07 15:46:52.899774 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.899755 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb"} err="failed to get container status \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": rpc error: code = NotFound desc = could not find container \"a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb\": container with ID starting with a8569d8a7e94da81f8c686bbf529dc5018358f599692354d2ec03531f6d513fb not found: ID does not exist" Jul 07 15:46:52.899816 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.899775 2585 scope.go:117] "RemoveContainer" containerID="a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f" Jul 07 15:46:52.900036 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900019 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f"} err="failed to get container status \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": rpc error: code = NotFound desc = could not find container \"a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f\": container with ID starting with a41cb1eab0e1a9b2c23576e060e8370d4df836fc41cd3a1c8122e4819d59b70f not found: ID does not exist" Jul 07 15:46:52.900104 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900036 2585 scope.go:117] "RemoveContainer" containerID="2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af" Jul 07 15:46:52.900220 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900207 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:46:52.900293 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900277 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af"} err="failed to get container status \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": rpc error: code = NotFound desc = could not find container \"2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af\": container with ID starting with 2c4c0087b4b34646038a1d1689d98e90cc315090d774593a93c264cd7ff9e7af not found: ID does not exist" Jul 07 15:46:52.900333 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900294 2585 scope.go:117] "RemoveContainer" containerID="2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53" Jul 07 15:46:52.900516 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900505 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-web" Jul 07 15:46:52.900550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900512 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53"} err="failed to get container status \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": rpc error: code = NotFound desc = could not find container \"2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53\": container with ID starting with 2acf5caac2b9e3c70be98e713ad360d8245b4eeb7a8d32acd2071e674de0db53 not found: ID does not exist" Jul 07 15:46:52.900550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900522 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-web" Jul 07 15:46:52.900550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900527 2585 scope.go:117] "RemoveContainer" containerID="6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655" Jul 07 15:46:52.900550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900534 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-thanos" Jul 07 15:46:52.900550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900540 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-thanos" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900553 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900560 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900574 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="thanos-sidecar" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900580 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="thanos-sidecar" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900588 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="init-config-reloader" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900593 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="init-config-reloader" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900598 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="config-reloader" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900606 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="config-reloader" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900622 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="prometheus" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900630 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="prometheus" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900691 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-web" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900700 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900706 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="config-reloader" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900705 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655"} err="failed to get container status \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": rpc error: code = NotFound desc = could not find container \"6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655\": container with ID starting with 6e76450ffa646a9f5f38584cc72d8589ed369869231ce6aa0ff639b13be85655 not found: ID does not exist" Jul 07 15:46:52.900719 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900712 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="kube-rbac-proxy-thanos" Jul 07 15:46:52.901185 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900731 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="thanos-sidecar" Jul 07 15:46:52.901185 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.900739 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" containerName="prometheus" Jul 07 15:46:52.905829 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.905813 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.908871 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.908851 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 15:46:52.908954 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.908922 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 15:46:52.909013 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.908878 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 15:46:52.909074 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.909025 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 15:46:52.909122 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.908902 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9hbv4\"" Jul 07 15:46:52.909187 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.908873 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 15:46:52.909286 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.909270 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 15:46:52.909457 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.909441 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 15:46:52.909662 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.909635 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 15:46:52.909951 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.909931 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 15:46:52.910048 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.909992 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 15:46:52.910342 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.910319 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 15:46:52.910406 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.910324 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 15:46:52.910870 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.910854 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4cm8hukvsm84v\"" Jul 07 15:46:52.913150 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.913129 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 15:46:52.914655 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.914638 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 15:46:52.919802 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.919779 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:46:52.948334 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948310 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-config\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948409 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948349 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948409 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948368 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948527 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948407 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948559 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948525 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/881fe027-2e23-4bfe-a2c6-0ea1281d8610-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948588 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948569 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948617 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948600 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948681 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948660 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948815 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948800 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948857 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948832 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948890 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948881 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-web-config\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.948926 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.948906 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/881fe027-2e23-4bfe-a2c6-0ea1281d8610-config-out\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949116 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949099 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949186 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949124 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949186 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949144 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949261 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949191 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949261 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949235 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949261 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhhv\" (UniqueName: \"kubernetes.io/projected/881fe027-2e23-4bfe-a2c6-0ea1281d8610-kube-api-access-fdhhv\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949408 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949338 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:52.949408 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:52.949381 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050098 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050013 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050098 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050044 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050098 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050066 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050098 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050109 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050138 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-web-config\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050187 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/881fe027-2e23-4bfe-a2c6-0ea1281d8610-config-out\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050238 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050292 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050314 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050341 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050361 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhhv\" (UniqueName: \"kubernetes.io/projected/881fe027-2e23-4bfe-a2c6-0ea1281d8610-kube-api-access-fdhhv\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050399 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.050439 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050428 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-config\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050479 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050508 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050532 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/881fe027-2e23-4bfe-a2c6-0ea1281d8610-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.050905 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.051102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.051075 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.053903 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053453 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.053903 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053492 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/881fe027-2e23-4bfe-a2c6-0ea1281d8610-config-out\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.053903 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053567 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/881fe027-2e23-4bfe-a2c6-0ea1281d8610-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.053903 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053612 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-web-config\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.053903 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053753 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.054252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053995 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.054252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.054062 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.054252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.054181 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.054252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.053567 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.054449 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.054430 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.054996 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.054967 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.055204 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.055184 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.055475 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.055454 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.056070 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.056048 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-config\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.056777 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.056752 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.056883 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.056860 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/881fe027-2e23-4bfe-a2c6-0ea1281d8610-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.057067 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.057051 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/881fe027-2e23-4bfe-a2c6-0ea1281d8610-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.060437 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.060415 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhhv\" (UniqueName: \"kubernetes.io/projected/881fe027-2e23-4bfe-a2c6-0ea1281d8610-kube-api-access-fdhhv\") pod \"prometheus-k8s-0\" (UID: \"881fe027-2e23-4bfe-a2c6-0ea1281d8610\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.216970 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.216930 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:53.358113 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.358092 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:46:53.359676 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:46:53.359653 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881fe027_2e23_4bfe_a2c6_0ea1281d8610.slice/crio-4f80932dba56abd7a6ef63a5759fd46aaf6f20d5033f122211a84b047fa9ccf5 WatchSource:0}: Error finding container 4f80932dba56abd7a6ef63a5759fd46aaf6f20d5033f122211a84b047fa9ccf5: Status 404 returned error can't find the container with id 4f80932dba56abd7a6ef63a5759fd46aaf6f20d5033f122211a84b047fa9ccf5 Jul 07 15:46:53.851597 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.851562 2585 generic.go:356] "Generic (PLEG): container finished" podID="881fe027-2e23-4bfe-a2c6-0ea1281d8610" containerID="493dd8c5bb332d0dcacb995f7a1cbddd9f0e9a9d7ac09f658848a7d8e9eaadd7" exitCode=0 Jul 07 15:46:53.852029 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.851644 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerDied","Data":"493dd8c5bb332d0dcacb995f7a1cbddd9f0e9a9d7ac09f658848a7d8e9eaadd7"} Jul 07 15:46:53.852029 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:53.851749 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"4f80932dba56abd7a6ef63a5759fd46aaf6f20d5033f122211a84b047fa9ccf5"} Jul 07 15:46:54.029922 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.029792 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf41a1d-72e5-4f33-9341-8426efeec099" path="/var/lib/kubelet/pods/eaf41a1d-72e5-4f33-9341-8426efeec099/volumes" Jul 07 15:46:54.860630 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.860596 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"f07582275de1a3d589cdb5027509868ee4a0e1b15a8b9920031c47e6fe2c0276"} Jul 07 15:46:54.861040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.860660 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"3cff6c167ce28ba5ef0ed582e268ac1480bf69c88718952dec8e2fd5e857ad15"} Jul 07 15:46:54.861040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.860686 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"b5bb9a4c636fd0c9f9783b56cbd79e744b357e406c391293a53c0e76bd16e48f"} Jul 07 15:46:54.861040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.860714 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"88d351a0672352b2d0768db1752690a2561727ec64e9b23ff79d6b7da242d0bc"} Jul 07 15:46:54.861040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.860738 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"614d6425ada509e2682154604f6e3c94d4a285e1f673956acf513c47cf99b770"} Jul 07 15:46:54.861040 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.860763 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881fe027-2e23-4bfe-a2c6-0ea1281d8610","Type":"ContainerStarted","Data":"51fa0b8e16086276d24aad5db2bfa29b9208a2b6aa3f61cbf6272d58e786ca9a"} Jul 07 15:46:54.891526 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:54.891472 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.872386109 podStartE2EDuration="2.891450709s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.0646ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:54.889315419 +0000 UTC m=+149.381033882" watchObservedRunningTime="2026-07-07 15:46:54.891450709 +0000 UTC m=+149.383169172" Jul 07 15:46:58.217425 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:46:58.217388 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:47:53.217807 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:47:53.217767 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:47:53.233852 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:47:53.233825 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:47:54.120253 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:47:54.120228 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:49:10.229638 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.229565 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8"] Jul 07 15:49:10.232368 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.232350 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.235092 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.235063 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 15:49:10.235241 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.235072 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 15:49:10.241245 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.241224 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8"] Jul 07 15:49:10.317214 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.317184 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkxhc\" (UniqueName: \"kubernetes.io/projected/96483da0-729f-4908-bdfc-977269992c1b-kube-api-access-pkxhc\") pod \"cert-manager-cainjector-559cdd8c87-xtlt8\" (UID: \"96483da0-729f-4908-bdfc-977269992c1b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.317352 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.317246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96483da0-729f-4908-bdfc-977269992c1b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xtlt8\" (UID: \"96483da0-729f-4908-bdfc-977269992c1b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.418150 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.418122 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96483da0-729f-4908-bdfc-977269992c1b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xtlt8\" (UID: \"96483da0-729f-4908-bdfc-977269992c1b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.418302 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.418201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkxhc\" (UniqueName: \"kubernetes.io/projected/96483da0-729f-4908-bdfc-977269992c1b-kube-api-access-pkxhc\") pod \"cert-manager-cainjector-559cdd8c87-xtlt8\" (UID: \"96483da0-729f-4908-bdfc-977269992c1b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.428329 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.428302 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96483da0-729f-4908-bdfc-977269992c1b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xtlt8\" (UID: \"96483da0-729f-4908-bdfc-977269992c1b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.428427 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.428365 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkxhc\" (UniqueName: \"kubernetes.io/projected/96483da0-729f-4908-bdfc-977269992c1b-kube-api-access-pkxhc\") pod \"cert-manager-cainjector-559cdd8c87-xtlt8\" (UID: \"96483da0-729f-4908-bdfc-977269992c1b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.555184 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.555146 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" Jul 07 15:49:10.682945 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:10.682925 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8"] Jul 07 15:49:10.685386 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:49:10.685353 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96483da0_729f_4908_bdfc_977269992c1b.slice/crio-fbfcdae6fce936e264c0f5ca3ead3b476d21ed2ed6fbefc7e0ae57b0320b2299 WatchSource:0}: Error finding container fbfcdae6fce936e264c0f5ca3ead3b476d21ed2ed6fbefc7e0ae57b0320b2299: Status 404 returned error can't find the container with id fbfcdae6fce936e264c0f5ca3ead3b476d21ed2ed6fbefc7e0ae57b0320b2299 Jul 07 15:49:11.420057 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:11.420013 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" event={"ID":"96483da0-729f-4908-bdfc-977269992c1b","Type":"ContainerStarted","Data":"fbfcdae6fce936e264c0f5ca3ead3b476d21ed2ed6fbefc7e0ae57b0320b2299"} Jul 07 15:49:14.434415 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:14.434375 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" event={"ID":"96483da0-729f-4908-bdfc-977269992c1b","Type":"ContainerStarted","Data":"13b66d1c21c0f2314d91fecc9483c29ff922c01e8a79450c5bb703ff04ccd5f9"} Jul 07 15:49:14.485460 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:14.485407 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtlt8" podStartSLOduration=1.67850661 podStartE2EDuration="4.485391091s" totalImagesPullingTime="2.806884481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:14.483811327 +0000 UTC m=+288.975529792" watchObservedRunningTime="2026-07-07 15:49:14.485391091 +0000 UTC m=+288.977109554" Jul 07 15:49:20.989682 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:20.989651 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p"] Jul 07 15:49:20.995637 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:20.995621 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:20.998507 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:20.998486 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 15:49:20.998620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:20.998512 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:49:20.998620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:20.998515 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cn8ln\"" Jul 07 15:49:21.002821 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.002800 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p"] Jul 07 15:49:21.112919 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.112890 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83cf9eb3-5f10-4be4-884f-e46404696d88-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jwf6p\" (UID: \"83cf9eb3-5f10-4be4-884f-e46404696d88\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.113086 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.112929 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzzsl\" (UniqueName: \"kubernetes.io/projected/83cf9eb3-5f10-4be4-884f-e46404696d88-kube-api-access-nzzsl\") pod \"openshift-lws-operator-fd8ccff4c-jwf6p\" (UID: \"83cf9eb3-5f10-4be4-884f-e46404696d88\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.214396 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.214363 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83cf9eb3-5f10-4be4-884f-e46404696d88-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jwf6p\" (UID: \"83cf9eb3-5f10-4be4-884f-e46404696d88\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.214396 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.214401 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzzsl\" (UniqueName: \"kubernetes.io/projected/83cf9eb3-5f10-4be4-884f-e46404696d88-kube-api-access-nzzsl\") pod \"openshift-lws-operator-fd8ccff4c-jwf6p\" (UID: \"83cf9eb3-5f10-4be4-884f-e46404696d88\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.214735 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.214717 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83cf9eb3-5f10-4be4-884f-e46404696d88-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jwf6p\" (UID: \"83cf9eb3-5f10-4be4-884f-e46404696d88\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.223611 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.223580 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzzsl\" (UniqueName: \"kubernetes.io/projected/83cf9eb3-5f10-4be4-884f-e46404696d88-kube-api-access-nzzsl\") pod \"openshift-lws-operator-fd8ccff4c-jwf6p\" (UID: \"83cf9eb3-5f10-4be4-884f-e46404696d88\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.306286 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.306225 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" Jul 07 15:49:21.442798 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.442772 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p"] Jul 07 15:49:21.445552 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:49:21.445526 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83cf9eb3_5f10_4be4_884f_e46404696d88.slice/crio-4fd9be34486e88d05f08d625441f03248c459117fc04026edca4c6b14c8a3625 WatchSource:0}: Error finding container 4fd9be34486e88d05f08d625441f03248c459117fc04026edca4c6b14c8a3625: Status 404 returned error can't find the container with id 4fd9be34486e88d05f08d625441f03248c459117fc04026edca4c6b14c8a3625 Jul 07 15:49:21.463938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:21.463912 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" event={"ID":"83cf9eb3-5f10-4be4-884f-e46404696d88","Type":"ContainerStarted","Data":"4fd9be34486e88d05f08d625441f03248c459117fc04026edca4c6b14c8a3625"} Jul 07 15:49:24.488876 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:24.488846 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" event={"ID":"83cf9eb3-5f10-4be4-884f-e46404696d88","Type":"ContainerStarted","Data":"033192391ed886ac41d20cd1a87d6a8abc10c488b0473e8ec9d47b8ae0b27e65"} Jul 07 15:49:24.507896 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:24.507850 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jwf6p" podStartSLOduration=1.841093725 podStartE2EDuration="4.507835094s" totalImagesPullingTime="2.666741369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:24.506881544 +0000 UTC m=+298.998600006" watchObservedRunningTime="2026-07-07 15:49:24.507835094 +0000 UTC m=+298.999553557" Jul 07 15:49:25.916886 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:25.916861 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 15:49:41.235639 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.235605 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn"] Jul 07 15:49:41.238537 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.238519 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.241583 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.241560 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 15:49:41.241583 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.241572 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 15:49:41.241811 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.241796 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 15:49:41.242068 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.242052 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hzz2w\"" Jul 07 15:49:41.242534 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.242521 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 15:49:41.256837 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.256817 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn"] Jul 07 15:49:41.289042 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.289017 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6adaf563-7f1f-44ff-835b-025ce7cd5a53-tmp\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.289202 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.289053 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn6zd\" (UniqueName: \"kubernetes.io/projected/6adaf563-7f1f-44ff-835b-025ce7cd5a53-kube-api-access-tn6zd\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.289202 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.289135 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6adaf563-7f1f-44ff-835b-025ce7cd5a53-manifests\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.289202 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.289172 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6adaf563-7f1f-44ff-835b-025ce7cd5a53-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.289325 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.289210 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6adaf563-7f1f-44ff-835b-025ce7cd5a53-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.390290 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.390260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6adaf563-7f1f-44ff-835b-025ce7cd5a53-manifests\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.390290 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.390291 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6adaf563-7f1f-44ff-835b-025ce7cd5a53-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.390451 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.390308 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6adaf563-7f1f-44ff-835b-025ce7cd5a53-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.390451 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.390372 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6adaf563-7f1f-44ff-835b-025ce7cd5a53-tmp\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.390451 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.390389 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6zd\" (UniqueName: \"kubernetes.io/projected/6adaf563-7f1f-44ff-835b-025ce7cd5a53-kube-api-access-tn6zd\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.390709 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.390685 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6adaf563-7f1f-44ff-835b-025ce7cd5a53-manifests\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.392718 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.392697 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6adaf563-7f1f-44ff-835b-025ce7cd5a53-tmp\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.392839 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.392803 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6adaf563-7f1f-44ff-835b-025ce7cd5a53-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.392948 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.392929 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6adaf563-7f1f-44ff-835b-025ce7cd5a53-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.398872 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.398852 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn6zd\" (UniqueName: \"kubernetes.io/projected/6adaf563-7f1f-44ff-835b-025ce7cd5a53-kube-api-access-tn6zd\") pod \"opendatahub-operator-controller-manager-56f57b78d4-2cszn\" (UID: \"6adaf563-7f1f-44ff-835b-025ce7cd5a53\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.548607 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.548523 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:41.682146 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.682123 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn"] Jul 07 15:49:41.684935 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:49:41.684909 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adaf563_7f1f_44ff_835b_025ce7cd5a53.slice/crio-06af1256719c2ff630442a72e7b8e6c096e19cec0977123383e6716d7dd8c2fd WatchSource:0}: Error finding container 06af1256719c2ff630442a72e7b8e6c096e19cec0977123383e6716d7dd8c2fd: Status 404 returned error can't find the container with id 06af1256719c2ff630442a72e7b8e6c096e19cec0977123383e6716d7dd8c2fd Jul 07 15:49:41.687237 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:41.687222 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:49:42.561762 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:42.561719 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" event={"ID":"6adaf563-7f1f-44ff-835b-025ce7cd5a53","Type":"ContainerStarted","Data":"06af1256719c2ff630442a72e7b8e6c096e19cec0977123383e6716d7dd8c2fd"} Jul 07 15:49:45.578229 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:45.578190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" event={"ID":"6adaf563-7f1f-44ff-835b-025ce7cd5a53","Type":"ContainerStarted","Data":"953ab44d86ed04640adebf0d2810675ca178c839dc8559a85c288cf1524d2ee3"} Jul 07 15:49:46.585957 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:46.585918 2585 generic.go:356] "Generic (PLEG): container finished" podID="6adaf563-7f1f-44ff-835b-025ce7cd5a53" containerID="953ab44d86ed04640adebf0d2810675ca178c839dc8559a85c288cf1524d2ee3" exitCode=0 Jul 07 15:49:46.586398 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:46.586010 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" event={"ID":"6adaf563-7f1f-44ff-835b-025ce7cd5a53","Type":"ContainerDied","Data":"953ab44d86ed04640adebf0d2810675ca178c839dc8559a85c288cf1524d2ee3"} Jul 07 15:49:47.592227 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:47.592195 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" event={"ID":"6adaf563-7f1f-44ff-835b-025ce7cd5a53","Type":"ContainerStarted","Data":"558f2ca25b7a23b4a1a616b73a474907ac56eb46a2bc8fb112766eebe8f401a4"} Jul 07 15:49:47.592589 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:47.592328 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:47.620955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:47.620905 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" podStartSLOduration=2.179177568 podStartE2EDuration="6.62089054s" totalImagesPullingTime="3.36917129s" totalInitContainerRuntime="1.072541682s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:47.619634991 +0000 UTC m=+322.111353456" watchObservedRunningTime="2026-07-07 15:49:47.62089054 +0000 UTC m=+322.112609001" Jul 07 15:49:58.598293 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:58.598264 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-2cszn" Jul 07 15:49:59.925600 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:49:59.925566 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv"] Jul 07 15:50:00.039074 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.039041 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv"] Jul 07 15:50:00.039275 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.039177 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.043970 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.043945 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vxprj\"" Jul 07 15:50:00.044290 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.044248 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 15:50:00.044403 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.044307 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 15:50:00.044403 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.044387 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 15:50:00.150554 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.150516 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-cert\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.150725 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.150566 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-metrics-cert\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.150725 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.150625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4s46\" (UniqueName: \"kubernetes.io/projected/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-kube-api-access-k4s46\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.150803 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.150722 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-manager-config\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.252216 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.252155 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-cert\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.252401 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.252243 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-metrics-cert\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.252401 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.252290 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4s46\" (UniqueName: \"kubernetes.io/projected/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-kube-api-access-k4s46\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.252401 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.252357 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-manager-config\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.253264 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.253230 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-manager-config\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.254807 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.254779 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-metrics-cert\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.254920 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.254880 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-cert\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.266504 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.266479 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4s46\" (UniqueName: \"kubernetes.io/projected/ec0a02b6-47ee-46ae-9e2b-29c609c638a7-kube-api-access-k4s46\") pod \"lws-controller-manager-646f7cc9db-x4bzv\" (UID: \"ec0a02b6-47ee-46ae-9e2b-29c609c638a7\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.349755 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.349726 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:00.487341 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.487318 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv"] Jul 07 15:50:00.489149 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:50:00.489127 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0a02b6_47ee_46ae_9e2b_29c609c638a7.slice/crio-5b360875b9063e04acc0b4a4b0db2a32de1cafe9a818f8a62d70859c5fba852d WatchSource:0}: Error finding container 5b360875b9063e04acc0b4a4b0db2a32de1cafe9a818f8a62d70859c5fba852d: Status 404 returned error can't find the container with id 5b360875b9063e04acc0b4a4b0db2a32de1cafe9a818f8a62d70859c5fba852d Jul 07 15:50:00.646510 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.646432 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" event={"ID":"ec0a02b6-47ee-46ae-9e2b-29c609c638a7","Type":"ContainerStarted","Data":"5b360875b9063e04acc0b4a4b0db2a32de1cafe9a818f8a62d70859c5fba852d"} Jul 07 15:50:00.862276 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.862246 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh"] Jul 07 15:50:00.865497 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.865469 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:00.869593 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.869572 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 15:50:00.869721 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.869671 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 15:50:00.870115 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.870101 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zsln6\"" Jul 07 15:50:00.870208 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.870102 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 15:50:00.870329 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.870316 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 15:50:00.882593 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.882573 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh"] Jul 07 15:50:00.958726 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.958651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vstl\" (UniqueName: \"kubernetes.io/projected/3c010cab-e547-4c0d-ab2c-bf36468f51f6-kube-api-access-8vstl\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:00.959206 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.958733 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c010cab-e547-4c0d-ab2c-bf36468f51f6-tmp\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:00.959206 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:00.958837 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c010cab-e547-4c0d-ab2c-bf36468f51f6-tls-certs\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.059675 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.059643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c010cab-e547-4c0d-ab2c-bf36468f51f6-tmp\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.059902 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.059702 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c010cab-e547-4c0d-ab2c-bf36468f51f6-tls-certs\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.059902 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.059754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vstl\" (UniqueName: \"kubernetes.io/projected/3c010cab-e547-4c0d-ab2c-bf36468f51f6-kube-api-access-8vstl\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.062259 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.062228 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c010cab-e547-4c0d-ab2c-bf36468f51f6-tmp\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.062659 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.062638 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c010cab-e547-4c0d-ab2c-bf36468f51f6-tls-certs\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.069702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.069681 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vstl\" (UniqueName: \"kubernetes.io/projected/3c010cab-e547-4c0d-ab2c-bf36468f51f6-kube-api-access-8vstl\") pod \"kube-auth-proxy-59ffcfbcb9-l4glh\" (UID: \"3c010cab-e547-4c0d-ab2c-bf36468f51f6\") " pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.176962 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.176925 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" Jul 07 15:50:01.340694 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.340658 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh"] Jul 07 15:50:01.344035 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:50:01.343997 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c010cab_e547_4c0d_ab2c_bf36468f51f6.slice/crio-8542d355dec512616f419f4dc16312343e06ec73051269da7ef3e169eff7a2c6 WatchSource:0}: Error finding container 8542d355dec512616f419f4dc16312343e06ec73051269da7ef3e169eff7a2c6: Status 404 returned error can't find the container with id 8542d355dec512616f419f4dc16312343e06ec73051269da7ef3e169eff7a2c6 Jul 07 15:50:01.653581 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:01.653489 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" event={"ID":"3c010cab-e547-4c0d-ab2c-bf36468f51f6","Type":"ContainerStarted","Data":"8542d355dec512616f419f4dc16312343e06ec73051269da7ef3e169eff7a2c6"} Jul 07 15:50:02.660252 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:02.660205 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" event={"ID":"ec0a02b6-47ee-46ae-9e2b-29c609c638a7","Type":"ContainerStarted","Data":"b5e5d4e9706f9e570ae6fea2330060874a80d01b4da697a8285d0a78d273ff72"} Jul 07 15:50:02.660681 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:02.660309 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:50:02.679786 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:02.679731 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" podStartSLOduration=2.440921983 podStartE2EDuration="3.679712915s" totalImagesPullingTime="1.238790932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:50:02.678912566 +0000 UTC m=+337.170631022" watchObservedRunningTime="2026-07-07 15:50:02.679712915 +0000 UTC m=+337.171431434" Jul 07 15:50:04.671358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:04.671322 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" event={"ID":"3c010cab-e547-4c0d-ab2c-bf36468f51f6","Type":"ContainerStarted","Data":"c226db039340bd52391e4c335fa9cb653c2018bd6d3f4b0c495ff9e4552af818"} Jul 07 15:50:04.692122 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:04.692064 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59ffcfbcb9-l4glh" podStartSLOduration=2.080769683 podStartE2EDuration="4.692045548s" totalImagesPullingTime="2.611275865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:50:04.691702564 +0000 UTC m=+339.183421026" watchObservedRunningTime="2026-07-07 15:50:04.692045548 +0000 UTC m=+339.183764015" Jul 07 15:50:13.668903 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:50:13.668875 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-x4bzv" Jul 07 15:51:50.220529 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.220445 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn"] Jul 07 15:51:50.223676 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.223656 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.226373 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.226354 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 15:51:50.226505 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.226397 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:51:50.226505 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.226397 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 15:51:50.227297 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.227283 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hmlvg\"" Jul 07 15:51:50.227297 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.227294 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:51:50.236320 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.236297 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn"] Jul 07 15:51:50.413729 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.413687 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1ecb404-786b-4e09-afe1-375c8b87cc44-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.413916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.413743 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1ecb404-786b-4e09-afe1-375c8b87cc44-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.413916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.413850 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9jk9\" (UniqueName: \"kubernetes.io/projected/e1ecb404-786b-4e09-afe1-375c8b87cc44-kube-api-access-g9jk9\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.514761 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.514731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9jk9\" (UniqueName: \"kubernetes.io/projected/e1ecb404-786b-4e09-afe1-375c8b87cc44-kube-api-access-g9jk9\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.514916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.514790 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1ecb404-786b-4e09-afe1-375c8b87cc44-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.514916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.514827 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1ecb404-786b-4e09-afe1-375c8b87cc44-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.515596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.515559 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1ecb404-786b-4e09-afe1-375c8b87cc44-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.517361 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.517341 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1ecb404-786b-4e09-afe1-375c8b87cc44-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.524406 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.524384 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9jk9\" (UniqueName: \"kubernetes.io/projected/e1ecb404-786b-4e09-afe1-375c8b87cc44-kube-api-access-g9jk9\") pod \"kuadrant-console-plugin-66b5774985-k5nhn\" (UID: \"e1ecb404-786b-4e09-afe1-375c8b87cc44\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.533879 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.533855 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" Jul 07 15:51:50.662648 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:50.662623 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn"] Jul 07 15:51:50.665095 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:51:50.665070 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1ecb404_786b_4e09_afe1_375c8b87cc44.slice/crio-c441d1ab64530369a06118d391c17a76a2190f7c2990a1cf3de4cced388a30ed WatchSource:0}: Error finding container c441d1ab64530369a06118d391c17a76a2190f7c2990a1cf3de4cced388a30ed: Status 404 returned error can't find the container with id c441d1ab64530369a06118d391c17a76a2190f7c2990a1cf3de4cced388a30ed Jul 07 15:51:51.171130 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:51.171097 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" event={"ID":"e1ecb404-786b-4e09-afe1-375c8b87cc44","Type":"ContainerStarted","Data":"c441d1ab64530369a06118d391c17a76a2190f7c2990a1cf3de4cced388a30ed"} Jul 07 15:51:59.244851 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.244820 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p"] Jul 07 15:51:59.251979 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.251953 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.255635 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.255614 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ttkqr\"" Jul 07 15:51:59.264144 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.264104 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p"] Jul 07 15:51:59.296032 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.295996 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf238285-4267-4c5b-8ac1-32bf6e3a590c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.296221 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.296050 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8652\" (UniqueName: \"kubernetes.io/projected/bf238285-4267-4c5b-8ac1-32bf6e3a590c-kube-api-access-c8652\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.397288 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.397249 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf238285-4267-4c5b-8ac1-32bf6e3a590c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.397288 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.397297 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8652\" (UniqueName: \"kubernetes.io/projected/bf238285-4267-4c5b-8ac1-32bf6e3a590c-kube-api-access-c8652\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.397679 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.397656 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf238285-4267-4c5b-8ac1-32bf6e3a590c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.407063 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.407038 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8652\" (UniqueName: \"kubernetes.io/projected/bf238285-4267-4c5b-8ac1-32bf6e3a590c-kube-api-access-c8652\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.565994 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.565912 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:51:59.940975 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.940876 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p"] Jul 07 15:51:59.943209 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:51:59.943184 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p"] Jul 07 15:52:00.018467 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.018088 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c"] Jul 07 15:52:00.027524 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.027482 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.055745 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.055614 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c"] Jul 07 15:52:00.105127 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.105098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2268c\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.105314 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.105275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s57v\" (UniqueName: \"kubernetes.io/projected/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-kube-api-access-6s57v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2268c\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.207100 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.206258 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s57v\" (UniqueName: \"kubernetes.io/projected/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-kube-api-access-6s57v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2268c\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.207100 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.206350 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2268c\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.207487 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.207438 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2268c\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.216249 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.216224 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s57v\" (UniqueName: \"kubernetes.io/projected/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-kube-api-access-6s57v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2268c\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:00.343113 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:00.343083 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:12.425963 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:12.425922 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c"] Jul 07 15:52:13.490675 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:13.490648 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c"] Jul 07 15:52:13.492115 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:52:13.492090 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacd9a1b_5ba4_4cd1_92e8_e851d96d55e3.slice/crio-e438334cd02e2d7fb642cfa4fd1c34f6157e459be64191b1c8151ecbbfb95286 WatchSource:0}: Error finding container e438334cd02e2d7fb642cfa4fd1c34f6157e459be64191b1c8151ecbbfb95286: Status 404 returned error can't find the container with id e438334cd02e2d7fb642cfa4fd1c34f6157e459be64191b1c8151ecbbfb95286 Jul 07 15:52:14.293476 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:14.293443 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" event={"ID":"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3","Type":"ContainerStarted","Data":"e438334cd02e2d7fb642cfa4fd1c34f6157e459be64191b1c8151ecbbfb95286"} Jul 07 15:52:14.294749 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:14.294728 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" event={"ID":"e1ecb404-786b-4e09-afe1-375c8b87cc44","Type":"ContainerStarted","Data":"806f337da4ff67be27408dd997321e0691f820ba005ca109c829bb94f79d0ed2"} Jul 07 15:52:16.184214 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:16.184042 2585 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-s6d2p_kuadrant-system_bf238285-4267-4c5b-8ac1-32bf6e3a590c_0(d8def7f343c6131296fa0353960c46ee57ba36f133d6485068aede553286a9b0): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-s6d2p to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 07 15:52:16.184214 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:16.184122 2585 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-s6d2p_kuadrant-system_bf238285-4267-4c5b-8ac1-32bf6e3a590c_0(d8def7f343c6131296fa0353960c46ee57ba36f133d6485068aede553286a9b0): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-s6d2p to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:16.310500 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.310466 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:16.313308 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:16.313274 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" podUID="bf238285-4267-4c5b-8ac1-32bf6e3a590c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:16.337612 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.337588 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:16.340341 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:16.340313 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" podUID="bf238285-4267-4c5b-8ac1-32bf6e3a590c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:16.466416 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.466344 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8652\" (UniqueName: \"kubernetes.io/projected/bf238285-4267-4c5b-8ac1-32bf6e3a590c-kube-api-access-c8652\") pod \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " Jul 07 15:52:16.466573 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.466438 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf238285-4267-4c5b-8ac1-32bf6e3a590c-extensions-socket-volume\") pod \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\" (UID: \"bf238285-4267-4c5b-8ac1-32bf6e3a590c\") " Jul 07 15:52:16.466743 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.466718 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf238285-4267-4c5b-8ac1-32bf6e3a590c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf238285-4267-4c5b-8ac1-32bf6e3a590c" (UID: "bf238285-4267-4c5b-8ac1-32bf6e3a590c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:52:16.469102 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.469079 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf238285-4267-4c5b-8ac1-32bf6e3a590c-kube-api-access-c8652" (OuterVolumeSpecName: "kube-api-access-c8652") pod "bf238285-4267-4c5b-8ac1-32bf6e3a590c" (UID: "bf238285-4267-4c5b-8ac1-32bf6e3a590c"). InnerVolumeSpecName "kube-api-access-c8652". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:52:16.567353 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.567321 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8652\" (UniqueName: \"kubernetes.io/projected/bf238285-4267-4c5b-8ac1-32bf6e3a590c-kube-api-access-c8652\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:52:16.567353 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:16.567346 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf238285-4267-4c5b-8ac1-32bf6e3a590c-extensions-socket-volume\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:52:17.316053 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:17.316022 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:17.318587 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:17.318557 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" podUID="bf238285-4267-4c5b-8ac1-32bf6e3a590c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:17.325708 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:17.325680 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s6d2p\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" podUID="bf238285-4267-4c5b-8ac1-32bf6e3a590c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s6d2p" Jul 07 15:52:18.032411 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:18.032381 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf238285-4267-4c5b-8ac1-32bf6e3a590c" path="/var/lib/kubelet/pods/bf238285-4267-4c5b-8ac1-32bf6e3a590c/volumes" Jul 07 15:52:21.337328 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.337294 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" event={"ID":"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3","Type":"ContainerStarted","Data":"9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242"} Jul 07 15:52:21.337768 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.337349 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" podUID="dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" containerName="manager" containerID="cri-o://9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242" gracePeriod=10 Jul 07 15:52:21.337768 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.337421 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:21.357648 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.357598 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" podStartSLOduration=15.421319401 podStartE2EDuration="22.35758077s" totalImagesPullingTime="6.936261369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:52:21.357504879 +0000 UTC m=+475.849223342" watchObservedRunningTime="2026-07-07 15:52:21.35758077 +0000 UTC m=+475.849299236" Jul 07 15:52:21.358359 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.358328 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-k5nhn" podStartSLOduration=8.264030876 podStartE2EDuration="31.358317615s" totalImagesPullingTime="23.094286739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:52:14.322672898 +0000 UTC m=+468.814391363" watchObservedRunningTime="2026-07-07 15:52:21.358317615 +0000 UTC m=+475.850036079" Jul 07 15:52:21.588541 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.588486 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:21.714535 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.714503 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s57v\" (UniqueName: \"kubernetes.io/projected/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-kube-api-access-6s57v\") pod \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " Jul 07 15:52:21.714771 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.714591 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-extensions-socket-volume\") pod \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\" (UID: \"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3\") " Jul 07 15:52:21.715074 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.715040 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" (UID: "dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:52:21.716843 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.716821 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-kube-api-access-6s57v" (OuterVolumeSpecName: "kube-api-access-6s57v") pod "dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" (UID: "dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3"). InnerVolumeSpecName "kube-api-access-6s57v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:52:21.815489 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.815452 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s57v\" (UniqueName: \"kubernetes.io/projected/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-kube-api-access-6s57v\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:52:21.815489 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:21.815484 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3-extensions-socket-volume\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:52:22.343514 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.343418 2585 generic.go:356] "Generic (PLEG): container finished" podID="dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" containerID="9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242" exitCode=0 Jul 07 15:52:22.343514 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.343469 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" Jul 07 15:52:22.343514 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.343496 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" event={"ID":"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3","Type":"ContainerDied","Data":"9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242"} Jul 07 15:52:22.344017 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.343555 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c" event={"ID":"dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3","Type":"ContainerDied","Data":"e438334cd02e2d7fb642cfa4fd1c34f6157e459be64191b1c8151ecbbfb95286"} Jul 07 15:52:22.344017 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.343576 2585 scope.go:117] "RemoveContainer" containerID="9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242" Jul 07 15:52:22.352334 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.352311 2585 scope.go:117] "RemoveContainer" containerID="9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242" Jul 07 15:52:22.352594 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:52:22.352575 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242\": container with ID starting with 9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242 not found: ID does not exist" containerID="9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242" Jul 07 15:52:22.352635 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.352602 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242"} err="failed to get container status \"9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242\": rpc error: code = NotFound desc = could not find container \"9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242\": container with ID starting with 9b9b27cd897f1b580ca1c98034804fadbb22c74fae196f1f08844030681c7242 not found: ID does not exist" Jul 07 15:52:22.363105 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.363081 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c"] Jul 07 15:52:22.366052 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:22.366031 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2268c"] Jul 07 15:52:24.030644 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:24.030617 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" path="/var/lib/kubelet/pods/dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3/volumes" Jul 07 15:52:44.541306 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.541263 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:52:44.541955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.541670 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" containerName="manager" Jul 07 15:52:44.541955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.541686 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" containerName="manager" Jul 07 15:52:44.541955 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.541751 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dacd9a1b-5ba4-4cd1-92e8-e851d96d55e3" containerName="manager" Jul 07 15:52:44.594648 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.594625 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:52:44.594770 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.594720 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.597536 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.597521 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 15:52:44.609100 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.609075 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77acd136-c6f3-4f11-8d0e-2d082f3ba260-config-file\") pod \"limitador-limitador-64c94fd64f-kgpkm\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.609271 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.609142 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s6jk\" (UniqueName: \"kubernetes.io/projected/77acd136-c6f3-4f11-8d0e-2d082f3ba260-kube-api-access-7s6jk\") pod \"limitador-limitador-64c94fd64f-kgpkm\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.636651 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.636627 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:52:44.709992 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.709970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s6jk\" (UniqueName: \"kubernetes.io/projected/77acd136-c6f3-4f11-8d0e-2d082f3ba260-kube-api-access-7s6jk\") pod \"limitador-limitador-64c94fd64f-kgpkm\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.710115 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.710012 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77acd136-c6f3-4f11-8d0e-2d082f3ba260-config-file\") pod \"limitador-limitador-64c94fd64f-kgpkm\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.710643 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.710625 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77acd136-c6f3-4f11-8d0e-2d082f3ba260-config-file\") pod \"limitador-limitador-64c94fd64f-kgpkm\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.719225 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.719208 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s6jk\" (UniqueName: \"kubernetes.io/projected/77acd136-c6f3-4f11-8d0e-2d082f3ba260-kube-api-access-7s6jk\") pod \"limitador-limitador-64c94fd64f-kgpkm\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:44.905404 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:44.905318 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:45.055678 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.055630 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:52:45.058239 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:52:45.058213 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77acd136_c6f3_4f11_8d0e_2d082f3ba260.slice/crio-c5e2ea12bad9e80e7509f24e87265cfc34df7b52bd02789fff9dcc51a92d9105 WatchSource:0}: Error finding container c5e2ea12bad9e80e7509f24e87265cfc34df7b52bd02789fff9dcc51a92d9105: Status 404 returned error can't find the container with id c5e2ea12bad9e80e7509f24e87265cfc34df7b52bd02789fff9dcc51a92d9105 Jul 07 15:52:45.446938 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.446878 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" event={"ID":"77acd136-c6f3-4f11-8d0e-2d082f3ba260","Type":"ContainerStarted","Data":"c5e2ea12bad9e80e7509f24e87265cfc34df7b52bd02789fff9dcc51a92d9105"} Jul 07 15:52:45.579811 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.579785 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-lsrzv"] Jul 07 15:52:45.582916 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.582896 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:52:45.585561 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.585542 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mj9p5\"" Jul 07 15:52:45.591067 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.591048 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lsrzv"] Jul 07 15:52:45.618219 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.618191 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqmzm\" (UniqueName: \"kubernetes.io/projected/619a47ae-f12e-4b26-9cf1-bac0ef980c14-kube-api-access-tqmzm\") pod \"authorino-75d556bd77-lsrzv\" (UID: \"619a47ae-f12e-4b26-9cf1-bac0ef980c14\") " pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:52:45.719731 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.719643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqmzm\" (UniqueName: \"kubernetes.io/projected/619a47ae-f12e-4b26-9cf1-bac0ef980c14-kube-api-access-tqmzm\") pod \"authorino-75d556bd77-lsrzv\" (UID: \"619a47ae-f12e-4b26-9cf1-bac0ef980c14\") " pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:52:45.735748 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.735715 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqmzm\" (UniqueName: \"kubernetes.io/projected/619a47ae-f12e-4b26-9cf1-bac0ef980c14-kube-api-access-tqmzm\") pod \"authorino-75d556bd77-lsrzv\" (UID: \"619a47ae-f12e-4b26-9cf1-bac0ef980c14\") " pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:52:45.893959 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:45.893701 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:52:46.057308 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:46.057284 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lsrzv"] Jul 07 15:52:46.059790 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:52:46.059761 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod619a47ae_f12e_4b26_9cf1_bac0ef980c14.slice/crio-302f7661b8e1e3a4852418d70841d37bf10dc9d43484a0e60489c7abe94c6f9a WatchSource:0}: Error finding container 302f7661b8e1e3a4852418d70841d37bf10dc9d43484a0e60489c7abe94c6f9a: Status 404 returned error can't find the container with id 302f7661b8e1e3a4852418d70841d37bf10dc9d43484a0e60489c7abe94c6f9a Jul 07 15:52:46.453124 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:46.453043 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lsrzv" event={"ID":"619a47ae-f12e-4b26-9cf1-bac0ef980c14","Type":"ContainerStarted","Data":"302f7661b8e1e3a4852418d70841d37bf10dc9d43484a0e60489c7abe94c6f9a"} Jul 07 15:52:48.467506 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:48.467465 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" event={"ID":"77acd136-c6f3-4f11-8d0e-2d082f3ba260","Type":"ContainerStarted","Data":"18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337"} Jul 07 15:52:48.467947 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:48.467598 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:52:48.486878 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:48.486830 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" podStartSLOduration=1.674235916 podStartE2EDuration="4.486814191s" totalImagesPullingTime="2.812578275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:52:48.484610698 +0000 UTC m=+502.976329357" watchObservedRunningTime="2026-07-07 15:52:48.486814191 +0000 UTC m=+502.978532671" Jul 07 15:52:50.480644 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:50.480554 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lsrzv" event={"ID":"619a47ae-f12e-4b26-9cf1-bac0ef980c14","Type":"ContainerStarted","Data":"09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180"} Jul 07 15:52:50.496650 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:50.496579 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-lsrzv" podStartSLOduration=1.463322877 podStartE2EDuration="5.49656228s" totalImagesPullingTime="4.033239403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:52:50.495916598 +0000 UTC m=+504.987635063" watchObservedRunningTime="2026-07-07 15:52:50.49656228 +0000 UTC m=+504.988280742" Jul 07 15:52:59.474910 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:52:59.474878 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:53:01.627223 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:01.627192 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:53:01.627691 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:01.627392 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" podUID="77acd136-c6f3-4f11-8d0e-2d082f3ba260" containerName="limitador" containerID="cri-o://18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337" gracePeriod=30 Jul 07 15:53:02.179946 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.179910 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:53:02.271630 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.271590 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77acd136-c6f3-4f11-8d0e-2d082f3ba260-config-file\") pod \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " Jul 07 15:53:02.271831 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.271647 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s6jk\" (UniqueName: \"kubernetes.io/projected/77acd136-c6f3-4f11-8d0e-2d082f3ba260-kube-api-access-7s6jk\") pod \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\" (UID: \"77acd136-c6f3-4f11-8d0e-2d082f3ba260\") " Jul 07 15:53:02.271980 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.271956 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77acd136-c6f3-4f11-8d0e-2d082f3ba260-config-file" (OuterVolumeSpecName: "config-file") pod "77acd136-c6f3-4f11-8d0e-2d082f3ba260" (UID: "77acd136-c6f3-4f11-8d0e-2d082f3ba260"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:53:02.273850 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.273829 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77acd136-c6f3-4f11-8d0e-2d082f3ba260-kube-api-access-7s6jk" (OuterVolumeSpecName: "kube-api-access-7s6jk") pod "77acd136-c6f3-4f11-8d0e-2d082f3ba260" (UID: "77acd136-c6f3-4f11-8d0e-2d082f3ba260"). InnerVolumeSpecName "kube-api-access-7s6jk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:53:02.373384 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.373337 2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77acd136-c6f3-4f11-8d0e-2d082f3ba260-config-file\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:53:02.373384 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.373377 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s6jk\" (UniqueName: \"kubernetes.io/projected/77acd136-c6f3-4f11-8d0e-2d082f3ba260-kube-api-access-7s6jk\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:53:02.535997 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.535911 2585 generic.go:356] "Generic (PLEG): container finished" podID="77acd136-c6f3-4f11-8d0e-2d082f3ba260" containerID="18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337" exitCode=0 Jul 07 15:53:02.535997 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.535956 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" Jul 07 15:53:02.535997 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.535972 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" event={"ID":"77acd136-c6f3-4f11-8d0e-2d082f3ba260","Type":"ContainerDied","Data":"18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337"} Jul 07 15:53:02.536293 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.536016 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kgpkm" event={"ID":"77acd136-c6f3-4f11-8d0e-2d082f3ba260","Type":"ContainerDied","Data":"c5e2ea12bad9e80e7509f24e87265cfc34df7b52bd02789fff9dcc51a92d9105"} Jul 07 15:53:02.536293 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.536036 2585 scope.go:117] "RemoveContainer" containerID="18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337" Jul 07 15:53:02.545078 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.545054 2585 scope.go:117] "RemoveContainer" containerID="18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337" Jul 07 15:53:02.545417 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:02.545396 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337\": container with ID starting with 18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337 not found: ID does not exist" containerID="18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337" Jul 07 15:53:02.545478 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.545427 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337"} err="failed to get container status \"18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337\": rpc error: code = NotFound desc = could not find container \"18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337\": container with ID starting with 18719c14477d420bed50a2b352c124e372a6d70bff1b8904fd98a74eb3923337 not found: ID does not exist" Jul 07 15:53:02.556989 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.556961 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:53:02.563248 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:02.563226 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kgpkm"] Jul 07 15:53:04.030766 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:04.030735 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77acd136-c6f3-4f11-8d0e-2d082f3ba260" path="/var/lib/kubelet/pods/77acd136-c6f3-4f11-8d0e-2d082f3ba260/volumes" Jul 07 15:53:05.415151 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.415095 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj"] Jul 07 15:53:05.415663 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.415478 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77acd136-c6f3-4f11-8d0e-2d082f3ba260" containerName="limitador" Jul 07 15:53:05.415663 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.415490 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77acd136-c6f3-4f11-8d0e-2d082f3ba260" containerName="limitador" Jul 07 15:53:05.415663 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.415564 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77acd136-c6f3-4f11-8d0e-2d082f3ba260" containerName="limitador" Jul 07 15:53:05.418729 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.418708 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.421495 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.421474 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-twv5v\"" Jul 07 15:53:05.421626 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.421472 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 07 15:53:05.422661 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.422645 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 07 15:53:05.422771 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.422723 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 07 15:53:05.429659 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.429622 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj"] Jul 07 15:53:05.501894 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.501860 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpk6c\" (UniqueName: \"kubernetes.io/projected/a4d05c80-2fdc-4b1d-94f1-ca9befdd6684-kube-api-access-vpk6c\") pod \"postgres-69fb4594cf-9dtqj\" (UID: \"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.502079 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.501917 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a4d05c80-2fdc-4b1d-94f1-ca9befdd6684-data\") pod \"postgres-69fb4594cf-9dtqj\" (UID: \"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.602844 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.602805 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpk6c\" (UniqueName: \"kubernetes.io/projected/a4d05c80-2fdc-4b1d-94f1-ca9befdd6684-kube-api-access-vpk6c\") pod \"postgres-69fb4594cf-9dtqj\" (UID: \"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.603029 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.602867 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a4d05c80-2fdc-4b1d-94f1-ca9befdd6684-data\") pod \"postgres-69fb4594cf-9dtqj\" (UID: \"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.603285 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.603270 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a4d05c80-2fdc-4b1d-94f1-ca9befdd6684-data\") pod \"postgres-69fb4594cf-9dtqj\" (UID: \"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.611712 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.611685 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpk6c\" (UniqueName: \"kubernetes.io/projected/a4d05c80-2fdc-4b1d-94f1-ca9befdd6684-kube-api-access-vpk6c\") pod \"postgres-69fb4594cf-9dtqj\" (UID: \"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.731527 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.731444 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:05.867528 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:05.867498 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj"] Jul 07 15:53:05.869524 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:53:05.869489 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d05c80_2fdc_4b1d_94f1_ca9befdd6684.slice/crio-2afdca481012b36e7a3bd4f1129523ef2b621fb165c85090ea91a4e9c8dbd9c4 WatchSource:0}: Error finding container 2afdca481012b36e7a3bd4f1129523ef2b621fb165c85090ea91a4e9c8dbd9c4: Status 404 returned error can't find the container with id 2afdca481012b36e7a3bd4f1129523ef2b621fb165c85090ea91a4e9c8dbd9c4 Jul 07 15:53:06.555914 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:06.555878 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" event={"ID":"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684","Type":"ContainerStarted","Data":"2afdca481012b36e7a3bd4f1129523ef2b621fb165c85090ea91a4e9c8dbd9c4"} Jul 07 15:53:11.593297 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:11.593258 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" event={"ID":"a4d05c80-2fdc-4b1d-94f1-ca9befdd6684","Type":"ContainerStarted","Data":"7a9fb6c66242aa4090e88a272cad56e2d804ee57fcf26fcf6ccadb0d6388d440"} Jul 07 15:53:11.593721 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:11.593393 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:11.610901 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:11.610849 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" podStartSLOduration=1.6157614 podStartE2EDuration="6.610834595s" totalImagesPullingTime="4.995073195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:53:11.610104995 +0000 UTC m=+526.101823460" watchObservedRunningTime="2026-07-07 15:53:11.610834595 +0000 UTC m=+526.102553086" Jul 07 15:53:17.628096 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:17.628026 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dtqj" Jul 07 15:53:18.159827 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.159791 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cvsvc"] Jul 07 15:53:18.165814 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.165790 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:18.171354 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.171331 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cvsvc"] Jul 07 15:53:18.319247 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.319204 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs68m\" (UniqueName: \"kubernetes.io/projected/bb8ba100-dd9d-410a-9199-072d7ad2cddc-kube-api-access-vs68m\") pod \"authorino-5d4b67c446-cvsvc\" (UID: \"bb8ba100-dd9d-410a-9199-072d7ad2cddc\") " pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:18.360942 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.360906 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cvsvc"] Jul 07 15:53:18.361232 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:18.361212 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vs68m], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-cvsvc" podUID="bb8ba100-dd9d-410a-9199-072d7ad2cddc" Jul 07 15:53:18.420078 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.419990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs68m\" (UniqueName: \"kubernetes.io/projected/bb8ba100-dd9d-410a-9199-072d7ad2cddc-kube-api-access-vs68m\") pod \"authorino-5d4b67c446-cvsvc\" (UID: \"bb8ba100-dd9d-410a-9199-072d7ad2cddc\") " pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:18.431564 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.431537 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs68m\" (UniqueName: \"kubernetes.io/projected/bb8ba100-dd9d-410a-9199-072d7ad2cddc-kube-api-access-vs68m\") pod \"authorino-5d4b67c446-cvsvc\" (UID: \"bb8ba100-dd9d-410a-9199-072d7ad2cddc\") " pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:18.626952 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.626919 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:18.633810 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.633789 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:18.722742 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.722662 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs68m\" (UniqueName: \"kubernetes.io/projected/bb8ba100-dd9d-410a-9199-072d7ad2cddc-kube-api-access-vs68m\") pod \"bb8ba100-dd9d-410a-9199-072d7ad2cddc\" (UID: \"bb8ba100-dd9d-410a-9199-072d7ad2cddc\") " Jul 07 15:53:18.724817 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.724788 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8ba100-dd9d-410a-9199-072d7ad2cddc-kube-api-access-vs68m" (OuterVolumeSpecName: "kube-api-access-vs68m") pod "bb8ba100-dd9d-410a-9199-072d7ad2cddc" (UID: "bb8ba100-dd9d-410a-9199-072d7ad2cddc"). InnerVolumeSpecName "kube-api-access-vs68m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:53:18.823953 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:18.823916 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs68m\" (UniqueName: \"kubernetes.io/projected/bb8ba100-dd9d-410a-9199-072d7ad2cddc-kube-api-access-vs68m\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:53:19.631992 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:19.631963 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cvsvc" Jul 07 15:53:19.664415 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:19.664390 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cvsvc"] Jul 07 15:53:19.668350 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:19.668324 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cvsvc"] Jul 07 15:53:19.756758 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:19.756727 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lsrzv"] Jul 07 15:53:19.756945 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:19.756922 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-lsrzv" podUID="619a47ae-f12e-4b26-9cf1-bac0ef980c14" containerName="authorino" containerID="cri-o://09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180" gracePeriod=30 Jul 07 15:53:20.013023 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.013000 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:53:20.030725 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.030691 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8ba100-dd9d-410a-9199-072d7ad2cddc" path="/var/lib/kubelet/pods/bb8ba100-dd9d-410a-9199-072d7ad2cddc/volumes" Jul 07 15:53:20.135772 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.135743 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqmzm\" (UniqueName: \"kubernetes.io/projected/619a47ae-f12e-4b26-9cf1-bac0ef980c14-kube-api-access-tqmzm\") pod \"619a47ae-f12e-4b26-9cf1-bac0ef980c14\" (UID: \"619a47ae-f12e-4b26-9cf1-bac0ef980c14\") " Jul 07 15:53:20.137909 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.137876 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619a47ae-f12e-4b26-9cf1-bac0ef980c14-kube-api-access-tqmzm" (OuterVolumeSpecName: "kube-api-access-tqmzm") pod "619a47ae-f12e-4b26-9cf1-bac0ef980c14" (UID: "619a47ae-f12e-4b26-9cf1-bac0ef980c14"). InnerVolumeSpecName "kube-api-access-tqmzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:53:20.237099 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.237064 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqmzm\" (UniqueName: \"kubernetes.io/projected/619a47ae-f12e-4b26-9cf1-bac0ef980c14-kube-api-access-tqmzm\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:53:20.637538 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.637454 2585 generic.go:356] "Generic (PLEG): container finished" podID="619a47ae-f12e-4b26-9cf1-bac0ef980c14" containerID="09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180" exitCode=0 Jul 07 15:53:20.637538 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.637495 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lsrzv" Jul 07 15:53:20.637716 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.637544 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lsrzv" event={"ID":"619a47ae-f12e-4b26-9cf1-bac0ef980c14","Type":"ContainerDied","Data":"09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180"} Jul 07 15:53:20.637716 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.637578 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lsrzv" event={"ID":"619a47ae-f12e-4b26-9cf1-bac0ef980c14","Type":"ContainerDied","Data":"302f7661b8e1e3a4852418d70841d37bf10dc9d43484a0e60489c7abe94c6f9a"} Jul 07 15:53:20.637716 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.637598 2585 scope.go:117] "RemoveContainer" containerID="09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180" Jul 07 15:53:20.646341 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.646323 2585 scope.go:117] "RemoveContainer" containerID="09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180" Jul 07 15:53:20.646563 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:20.646546 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180\": container with ID starting with 09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180 not found: ID does not exist" containerID="09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180" Jul 07 15:53:20.646604 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.646570 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180"} err="failed to get container status \"09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180\": rpc error: code = NotFound desc = could not find container \"09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180\": container with ID starting with 09393b64f1470489bb86dcdb31e54769f31cd72491d9b80c133b5b3979ca7180 not found: ID does not exist" Jul 07 15:53:20.658918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.658889 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lsrzv"] Jul 07 15:53:20.662826 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:20.662806 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lsrzv"] Jul 07 15:53:22.030898 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:22.030856 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619a47ae-f12e-4b26-9cf1-bac0ef980c14" path="/var/lib/kubelet/pods/619a47ae-f12e-4b26-9cf1-bac0ef980c14/volumes" Jul 07 15:53:24.072010 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.071975 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-588c6b8857-9fv94"] Jul 07 15:53:24.072434 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.072366 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a47ae-f12e-4b26-9cf1-bac0ef980c14" containerName="authorino" Jul 07 15:53:24.072434 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.072378 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a47ae-f12e-4b26-9cf1-bac0ef980c14" containerName="authorino" Jul 07 15:53:24.072549 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.072449 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="619a47ae-f12e-4b26-9cf1-bac0ef980c14" containerName="authorino" Jul 07 15:53:24.117553 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.117509 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588c6b8857-9fv94"] Jul 07 15:53:24.117732 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.117621 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.120458 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.120433 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 15:53:24.120620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.120480 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 15:53:24.120620 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.120551 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wk7g6\"" Jul 07 15:53:24.175993 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.175953 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmdk8\" (UniqueName: \"kubernetes.io/projected/6bf010b4-355e-45a8-9af7-42a719dad05e-kube-api-access-zmdk8\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.176182 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.176001 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.238751 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.238718 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-796b9b6788-bv9h5"] Jul 07 15:53:24.261390 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.261359 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796b9b6788-bv9h5"] Jul 07 15:53:24.261541 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.261465 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.276801 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.276774 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbls4\" (UniqueName: \"kubernetes.io/projected/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-kube-api-access-xbls4\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.276930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.276833 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmdk8\" (UniqueName: \"kubernetes.io/projected/6bf010b4-355e-45a8-9af7-42a719dad05e-kube-api-access-zmdk8\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.276930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.276896 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.277031 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.276965 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.277031 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:24.277024 2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 15:53:24.277280 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:24.277084 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert podName:6bf010b4-355e-45a8-9af7-42a719dad05e nodeName:}" failed. No retries permitted until 2026-07-07 15:53:24.777063639 +0000 UTC m=+539.268782093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert") pod "maas-controller-588c6b8857-9fv94" (UID: "6bf010b4-355e-45a8-9af7-42a719dad05e") : secret "maas-controller-webhook-cert" not found Jul 07 15:53:24.287444 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.287411 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmdk8\" (UniqueName: \"kubernetes.io/projected/6bf010b4-355e-45a8-9af7-42a719dad05e-kube-api-access-zmdk8\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.377550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.377465 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.377708 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.377553 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbls4\" (UniqueName: \"kubernetes.io/projected/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-kube-api-access-xbls4\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.377708 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:24.377619 2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 15:53:24.377708 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:24.377685 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert podName:9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02 nodeName:}" failed. No retries permitted until 2026-07-07 15:53:24.877668087 +0000 UTC m=+539.369386528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert") pod "maas-controller-796b9b6788-bv9h5" (UID: "9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02") : secret "maas-controller-webhook-cert" not found Jul 07 15:53:24.387241 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.387207 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbls4\" (UniqueName: \"kubernetes.io/projected/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-kube-api-access-xbls4\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.781117 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.781079 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.783686 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.783662 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert\") pod \"maas-controller-588c6b8857-9fv94\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:24.882198 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.882131 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:24.884747 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:24.884722 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert\") pod \"maas-controller-796b9b6788-bv9h5\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:25.030243 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:25.030210 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:25.168358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:25.168334 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588c6b8857-9fv94"] Jul 07 15:53:25.170353 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:53:25.170325 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf010b4_355e_45a8_9af7_42a719dad05e.slice/crio-c34123573fb5cc92c3532016ba5e879225df96843ab28fcd6e3f9705f4061461 WatchSource:0}: Error finding container c34123573fb5cc92c3532016ba5e879225df96843ab28fcd6e3f9705f4061461: Status 404 returned error can't find the container with id c34123573fb5cc92c3532016ba5e879225df96843ab28fcd6e3f9705f4061461 Jul 07 15:53:25.171333 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:25.171313 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:25.304143 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:25.304120 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796b9b6788-bv9h5"] Jul 07 15:53:25.306073 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:53:25.306045 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebcca26_4da4_49cc_9c0c_6eadd9fd5c02.slice/crio-8999d5aec513ba645aeddcf87fb3486fe136716a283e7192eec7d244060ad2ca WatchSource:0}: Error finding container 8999d5aec513ba645aeddcf87fb3486fe136716a283e7192eec7d244060ad2ca: Status 404 returned error can't find the container with id 8999d5aec513ba645aeddcf87fb3486fe136716a283e7192eec7d244060ad2ca Jul 07 15:53:25.661979 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:25.661880 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796b9b6788-bv9h5" event={"ID":"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02","Type":"ContainerStarted","Data":"8999d5aec513ba645aeddcf87fb3486fe136716a283e7192eec7d244060ad2ca"} Jul 07 15:53:25.662904 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:25.662883 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588c6b8857-9fv94" event={"ID":"6bf010b4-355e-45a8-9af7-42a719dad05e","Type":"ContainerStarted","Data":"c34123573fb5cc92c3532016ba5e879225df96843ab28fcd6e3f9705f4061461"} Jul 07 15:53:27.973930 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:27.973904 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 15:53:28.680369 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:28.680329 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796b9b6788-bv9h5" event={"ID":"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02","Type":"ContainerStarted","Data":"cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12"} Jul 07 15:53:28.680584 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:28.680396 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:28.681701 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:28.681676 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588c6b8857-9fv94" event={"ID":"6bf010b4-355e-45a8-9af7-42a719dad05e","Type":"ContainerStarted","Data":"a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726"} Jul 07 15:53:28.681824 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:28.681807 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:28.706487 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:28.706431 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-796b9b6788-bv9h5" podStartSLOduration=2.035126125 podStartE2EDuration="4.706414176s" totalImagesPullingTime="2.671288051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:53:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:53:28.704213856 +0000 UTC m=+543.195932321" watchObservedRunningTime="2026-07-07 15:53:28.706414176 +0000 UTC m=+543.198132639" Jul 07 15:53:28.724551 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:28.724489 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-588c6b8857-9fv94" podStartSLOduration=1.925095606 podStartE2EDuration="4.724474755s" totalImagesPullingTime="2.799379149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:53:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:53:28.723298377 +0000 UTC m=+543.215016841" watchObservedRunningTime="2026-07-07 15:53:28.724474755 +0000 UTC m=+543.216193217" Jul 07 15:53:39.694154 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:39.694119 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:39.694625 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:39.694264 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:53:39.773596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:39.773552 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588c6b8857-9fv94"] Jul 07 15:53:39.773819 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:39.773784 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-588c6b8857-9fv94" podUID="6bf010b4-355e-45a8-9af7-42a719dad05e" containerName="manager" containerID="cri-o://a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726" gracePeriod=30 Jul 07 15:53:40.024608 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.024581 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:40.118424 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.118387 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmdk8\" (UniqueName: \"kubernetes.io/projected/6bf010b4-355e-45a8-9af7-42a719dad05e-kube-api-access-zmdk8\") pod \"6bf010b4-355e-45a8-9af7-42a719dad05e\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " Jul 07 15:53:40.118627 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.118436 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert\") pod \"6bf010b4-355e-45a8-9af7-42a719dad05e\" (UID: \"6bf010b4-355e-45a8-9af7-42a719dad05e\") " Jul 07 15:53:40.120873 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.120839 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf010b4-355e-45a8-9af7-42a719dad05e-kube-api-access-zmdk8" (OuterVolumeSpecName: "kube-api-access-zmdk8") pod "6bf010b4-355e-45a8-9af7-42a719dad05e" (UID: "6bf010b4-355e-45a8-9af7-42a719dad05e"). InnerVolumeSpecName "kube-api-access-zmdk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:53:40.120873 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.120852 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6bf010b4-355e-45a8-9af7-42a719dad05e" (UID: "6bf010b4-355e-45a8-9af7-42a719dad05e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:53:40.220121 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.220042 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmdk8\" (UniqueName: \"kubernetes.io/projected/6bf010b4-355e-45a8-9af7-42a719dad05e-kube-api-access-zmdk8\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:53:40.220121 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.220075 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf010b4-355e-45a8-9af7-42a719dad05e-webhook-cert\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:53:40.740114 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.740078 2585 generic.go:356] "Generic (PLEG): container finished" podID="6bf010b4-355e-45a8-9af7-42a719dad05e" containerID="a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726" exitCode=0 Jul 07 15:53:40.740596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.740128 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588c6b8857-9fv94" Jul 07 15:53:40.740596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.740194 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588c6b8857-9fv94" event={"ID":"6bf010b4-355e-45a8-9af7-42a719dad05e","Type":"ContainerDied","Data":"a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726"} Jul 07 15:53:40.740596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.740253 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588c6b8857-9fv94" event={"ID":"6bf010b4-355e-45a8-9af7-42a719dad05e","Type":"ContainerDied","Data":"c34123573fb5cc92c3532016ba5e879225df96843ab28fcd6e3f9705f4061461"} Jul 07 15:53:40.740596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.740290 2585 scope.go:117] "RemoveContainer" containerID="a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726" Jul 07 15:53:40.751870 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.751855 2585 scope.go:117] "RemoveContainer" containerID="a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726" Jul 07 15:53:40.752127 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:53:40.752110 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726\": container with ID starting with a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726 not found: ID does not exist" containerID="a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726" Jul 07 15:53:40.752221 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.752139 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726"} err="failed to get container status \"a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726\": rpc error: code = NotFound desc = could not find container \"a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726\": container with ID starting with a1d1ea5bc333e6a1cd27c049128dae8befed6144f3661436a6812b6b4a3c0726 not found: ID does not exist" Jul 07 15:53:40.766334 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.766314 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588c6b8857-9fv94"] Jul 07 15:53:40.770642 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:40.770622 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-588c6b8857-9fv94"] Jul 07 15:53:42.030285 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:53:42.030256 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf010b4-355e-45a8-9af7-42a719dad05e" path="/var/lib/kubelet/pods/6bf010b4-355e-45a8-9af7-42a719dad05e/volumes" Jul 07 15:54:01.051200 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.051147 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c85595867-sw24x"] Jul 07 15:54:01.052041 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.052020 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf010b4-355e-45a8-9af7-42a719dad05e" containerName="manager" Jul 07 15:54:01.052041 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.052043 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf010b4-355e-45a8-9af7-42a719dad05e" containerName="manager" Jul 07 15:54:01.052246 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.052136 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf010b4-355e-45a8-9af7-42a719dad05e" containerName="manager" Jul 07 15:54:01.054969 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.054947 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.057801 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.057782 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7b4t8\"" Jul 07 15:54:01.057918 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.057903 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 07 15:54:01.066016 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.065986 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c85595867-sw24x"] Jul 07 15:54:01.214400 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.214364 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-openshift-service-ca\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.214400 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.214407 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-maas-api-tls\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.214646 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.214442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jqkw\" (UniqueName: \"kubernetes.io/projected/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-kube-api-access-7jqkw\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.315044 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.314961 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jqkw\" (UniqueName: \"kubernetes.io/projected/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-kube-api-access-7jqkw\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.315228 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.315071 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-openshift-service-ca\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.315228 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.315113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-maas-api-tls\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.315720 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.315697 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-openshift-service-ca\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.317702 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.317680 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-maas-api-tls\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.324625 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.324600 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jqkw\" (UniqueName: \"kubernetes.io/projected/7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8-kube-api-access-7jqkw\") pod \"maas-api-c85595867-sw24x\" (UID: \"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8\") " pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.365305 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.365274 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:01.508766 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.508742 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c85595867-sw24x"] Jul 07 15:54:01.512060 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:54:01.512021 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad90a6a_64fc_4cee_a1c6_d81ce1ca9fe8.slice/crio-dd368081c04dcd7fc650f1562897c91c0a7983c748dcc4e2f92a4db91334f7c6 WatchSource:0}: Error finding container dd368081c04dcd7fc650f1562897c91c0a7983c748dcc4e2f92a4db91334f7c6: Status 404 returned error can't find the container with id dd368081c04dcd7fc650f1562897c91c0a7983c748dcc4e2f92a4db91334f7c6 Jul 07 15:54:01.839472 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:01.839432 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" event={"ID":"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8","Type":"ContainerStarted","Data":"dd368081c04dcd7fc650f1562897c91c0a7983c748dcc4e2f92a4db91334f7c6"} Jul 07 15:54:03.852258 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:03.852143 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" event={"ID":"7ad90a6a-64fc-4cee-a1c6-d81ce1ca9fe8","Type":"ContainerStarted","Data":"22c42715dd245f28392d52777708c727179cbbf33f92a5271ab28e7edf8a43e2"} Jul 07 15:54:03.852683 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:03.852337 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:03.876075 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:03.876025 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" podStartSLOduration=0.834266146 podStartE2EDuration="2.876008207s" totalImagesPullingTime="2.041742061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:03.874877423 +0000 UTC m=+578.366595888" watchObservedRunningTime="2026-07-07 15:54:03.876008207 +0000 UTC m=+578.367726671" Jul 07 15:54:09.864996 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:09.864969 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c85595867-sw24x" Jul 07 15:54:17.159381 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.159348 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2"] Jul 07 15:54:17.163280 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.163255 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.166180 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.166133 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 15:54:17.167262 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.167242 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 15:54:17.167354 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.167247 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vvllp\"" Jul 07 15:54:17.167354 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.167246 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 15:54:17.182779 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.182749 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2"] Jul 07 15:54:17.260444 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.260414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.260444 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.260445 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.260647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.260467 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ae2f79e-9951-4ee4-b98a-e93a0400014a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.260647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.260516 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5gtv\" (UniqueName: \"kubernetes.io/projected/3ae2f79e-9951-4ee4-b98a-e93a0400014a-kube-api-access-j5gtv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.260647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.260596 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.260647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.260641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362112 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362070 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362318 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362318 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362186 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362318 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362213 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ae2f79e-9951-4ee4-b98a-e93a0400014a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362318 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362249 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5gtv\" (UniqueName: \"kubernetes.io/projected/3ae2f79e-9951-4ee4-b98a-e93a0400014a-kube-api-access-j5gtv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362521 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362575 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362555 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362624 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362609 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.362685 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.362668 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.364671 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.364650 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ae2f79e-9951-4ee4-b98a-e93a0400014a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.364904 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.364886 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ae2f79e-9951-4ee4-b98a-e93a0400014a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.372490 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.372469 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5gtv\" (UniqueName: \"kubernetes.io/projected/3ae2f79e-9951-4ee4-b98a-e93a0400014a-kube-api-access-j5gtv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2\" (UID: \"3ae2f79e-9951-4ee4-b98a-e93a0400014a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.473434 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.473341 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:17.616332 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.616305 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2"] Jul 07 15:54:17.618678 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:54:17.618649 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae2f79e_9951_4ee4_b98a_e93a0400014a.slice/crio-0c894cc35e306b4b77725c0c4a4b2a790e2cd12d8b8c87457bf1b5b9239f0700 WatchSource:0}: Error finding container 0c894cc35e306b4b77725c0c4a4b2a790e2cd12d8b8c87457bf1b5b9239f0700: Status 404 returned error can't find the container with id 0c894cc35e306b4b77725c0c4a4b2a790e2cd12d8b8c87457bf1b5b9239f0700 Jul 07 15:54:17.924826 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:17.924790 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" event={"ID":"3ae2f79e-9951-4ee4-b98a-e93a0400014a","Type":"ContainerStarted","Data":"0c894cc35e306b4b77725c0c4a4b2a790e2cd12d8b8c87457bf1b5b9239f0700"} Jul 07 15:54:18.931520 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:18.931433 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" event={"ID":"3ae2f79e-9951-4ee4-b98a-e93a0400014a","Type":"ContainerStarted","Data":"fe1646415874458ce7cea65830f88cb131331daca850f0fd8bf6fe941e85c20b"} Jul 07 15:54:18.961859 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:18.961804 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" podStartSLOduration=0.928440064 podStartE2EDuration="1.961786063s" totalImagesPullingTime="1.033345999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:18.961609713 +0000 UTC m=+593.453328178" watchObservedRunningTime="2026-07-07 15:54:18.961786063 +0000 UTC m=+593.453504527" Jul 07 15:54:27.474150 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:27.474106 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:27.474577 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:27.474216 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:27.486846 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:27.486820 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:27.984668 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:27.984641 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2" Jul 07 15:54:37.025882 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.025843 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs"] Jul 07 15:54:37.029505 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.029481 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.036127 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.032605 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 15:54:37.042205 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.042180 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs"] Jul 07 15:54:37.151436 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.151396 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.151436 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.151438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hstxw\" (UniqueName: \"kubernetes.io/projected/a2842970-2e1f-4b20-b285-281e71feb6e6-kube-api-access-hstxw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.151697 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.151465 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.151697 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.151616 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.151813 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.151709 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2842970-2e1f-4b20-b285-281e71feb6e6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.151813 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.151741 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.252728 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.252688 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.252926 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.252750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2842970-2e1f-4b20-b285-281e71feb6e6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.252926 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.252867 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.252926 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.252923 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.253093 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.252944 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hstxw\" (UniqueName: \"kubernetes.io/projected/a2842970-2e1f-4b20-b285-281e71feb6e6-kube-api-access-hstxw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.253093 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.252973 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.253344 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.253313 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.253419 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.253391 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.253476 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.253425 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.255231 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.255202 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2842970-2e1f-4b20-b285-281e71feb6e6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.255404 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.255389 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2842970-2e1f-4b20-b285-281e71feb6e6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.262135 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.262113 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hstxw\" (UniqueName: \"kubernetes.io/projected/a2842970-2e1f-4b20-b285-281e71feb6e6-kube-api-access-hstxw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tgwvs\" (UID: \"a2842970-2e1f-4b20-b285-281e71feb6e6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.346962 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.346881 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:37.481488 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:37.481453 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs"] Jul 07 15:54:37.483608 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:54:37.483584 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2842970_2e1f_4b20_b285_281e71feb6e6.slice/crio-a6fb4c0b532f9409debce301a63c6a68faa30c8feb38cfc00b624ff243d4540e WatchSource:0}: Error finding container a6fb4c0b532f9409debce301a63c6a68faa30c8feb38cfc00b624ff243d4540e: Status 404 returned error can't find the container with id a6fb4c0b532f9409debce301a63c6a68faa30c8feb38cfc00b624ff243d4540e Jul 07 15:54:38.029944 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:38.029912 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" event={"ID":"a2842970-2e1f-4b20-b285-281e71feb6e6","Type":"ContainerStarted","Data":"c9f9ccceb629a3813be3e775bb3f3218574026fa6e55c09c81b69032e6cd4f40"} Jul 07 15:54:38.030358 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:38.029965 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" event={"ID":"a2842970-2e1f-4b20-b285-281e71feb6e6","Type":"ContainerStarted","Data":"a6fb4c0b532f9409debce301a63c6a68faa30c8feb38cfc00b624ff243d4540e"} Jul 07 15:54:38.058123 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:38.058062 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" podStartSLOduration=0.867337315 podStartE2EDuration="1.058044449s" totalImagesPullingTime="190.707134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:38.055798738 +0000 UTC m=+612.547517214" watchObservedRunningTime="2026-07-07 15:54:38.058044449 +0000 UTC m=+612.549762920" Jul 07 15:54:47.347596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:47.347508 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:47.347596 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:47.347569 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:47.360709 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:47.360683 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:54:48.091106 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:54:48.091079 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tgwvs" Jul 07 15:55:02.031696 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.031668 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2"] Jul 07 15:55:02.035538 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.035517 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.038227 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.038208 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 15:55:02.052896 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.052867 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2"] Jul 07 15:55:02.079412 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.079387 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.079550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.079417 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.079550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.079472 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg84b\" (UniqueName: \"kubernetes.io/projected/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-kube-api-access-kg84b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.079550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.079500 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.079550 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.079535 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.079750 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.079641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.180647 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.180609 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg84b\" (UniqueName: \"kubernetes.io/projected/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-kube-api-access-kg84b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.180843 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.180652 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.180843 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.180705 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.180843 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.180769 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.180843 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.180816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.180843 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.180836 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.181147 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.181114 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.181300 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.181187 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.181602 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.181570 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.183384 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.183357 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.183590 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.183571 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.190122 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.190093 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg84b\" (UniqueName: \"kubernetes.io/projected/4c08b0e5-2635-4ea7-bab6-b9c7031c075e-kube-api-access-kg84b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2\" (UID: \"4c08b0e5-2635-4ea7-bab6-b9c7031c075e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.350287 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.350214 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:02.492410 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.492383 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2"] Jul 07 15:55:02.493307 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:55:02.493282 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c08b0e5_2635_4ea7_bab6_b9c7031c075e.slice/crio-e813bb9f0e69fe659b3314ac4723b54365513e6b758727e058ea799f194b65f0 WatchSource:0}: Error finding container e813bb9f0e69fe659b3314ac4723b54365513e6b758727e058ea799f194b65f0: Status 404 returned error can't find the container with id e813bb9f0e69fe659b3314ac4723b54365513e6b758727e058ea799f194b65f0 Jul 07 15:55:02.494766 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:02.494745 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:55:03.150130 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:03.150040 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" event={"ID":"4c08b0e5-2635-4ea7-bab6-b9c7031c075e","Type":"ContainerStarted","Data":"4321bcb2b1c0661d3efa5d29892f0dff64472e3e25c9f970f025025e8e462d99"} Jul 07 15:55:03.150130 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:03.150093 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" event={"ID":"4c08b0e5-2635-4ea7-bab6-b9c7031c075e","Type":"ContainerStarted","Data":"e813bb9f0e69fe659b3314ac4723b54365513e6b758727e058ea799f194b65f0"} Jul 07 15:55:03.171806 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:03.171754 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" podStartSLOduration=0.925459391 podStartE2EDuration="1.171739987s" totalImagesPullingTime="246.280596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:55:03.170860961 +0000 UTC m=+637.662579425" watchObservedRunningTime="2026-07-07 15:55:03.171739987 +0000 UTC m=+637.663458449" Jul 07 15:55:12.351330 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:12.351291 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:12.351724 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:12.351371 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:12.363600 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:12.363572 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:55:13.210310 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:55:13.210284 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2" Jul 07 15:57:48.189058 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.188964 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-796b9b6788-bv9h5"] Jul 07 15:57:48.189642 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.189239 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-796b9b6788-bv9h5" podUID="9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" containerName="manager" containerID="cri-o://cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12" gracePeriod=30 Jul 07 15:57:48.434333 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.434310 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:57:48.596028 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.595998 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbls4\" (UniqueName: \"kubernetes.io/projected/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-kube-api-access-xbls4\") pod \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " Jul 07 15:57:48.596250 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.596040 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert\") pod \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\" (UID: \"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02\") " Jul 07 15:57:48.598280 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.598248 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" (UID: "9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:57:48.598394 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.598347 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-kube-api-access-xbls4" (OuterVolumeSpecName: "kube-api-access-xbls4") pod "9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" (UID: "9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02"). InnerVolumeSpecName "kube-api-access-xbls4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:57:48.697658 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.697605 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbls4\" (UniqueName: \"kubernetes.io/projected/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-kube-api-access-xbls4\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:57:48.697658 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.697652 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02-webhook-cert\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 15:57:48.939037 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.938941 2585 generic.go:356] "Generic (PLEG): container finished" podID="9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" containerID="cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12" exitCode=0 Jul 07 15:57:48.939037 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.938992 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796b9b6788-bv9h5" Jul 07 15:57:48.939037 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.939000 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796b9b6788-bv9h5" event={"ID":"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02","Type":"ContainerDied","Data":"cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12"} Jul 07 15:57:48.939037 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.939040 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796b9b6788-bv9h5" event={"ID":"9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02","Type":"ContainerDied","Data":"8999d5aec513ba645aeddcf87fb3486fe136716a283e7192eec7d244060ad2ca"} Jul 07 15:57:48.939382 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.939061 2585 scope.go:117] "RemoveContainer" containerID="cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12" Jul 07 15:57:48.950584 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.950564 2585 scope.go:117] "RemoveContainer" containerID="cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12" Jul 07 15:57:48.950870 ip-10-0-136-171 kubenswrapper[2585]: E0707 15:57:48.950849 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12\": container with ID starting with cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12 not found: ID does not exist" containerID="cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12" Jul 07 15:57:48.950925 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.950878 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12"} err="failed to get container status \"cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12\": rpc error: code = NotFound desc = could not find container \"cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12\": container with ID starting with cee815188d230abed4d55c50096215574c5d1d13532fd9ebf307e66931c52c12 not found: ID does not exist" Jul 07 15:57:48.964680 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.964650 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-796b9b6788-bv9h5"] Jul 07 15:57:48.967397 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:48.967375 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-796b9b6788-bv9h5"] Jul 07 15:57:49.789359 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.789326 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-796b9b6788-85hkb"] Jul 07 15:57:49.789784 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.789726 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" containerName="manager" Jul 07 15:57:49.789784 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.789738 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" containerName="manager" Jul 07 15:57:49.789863 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.789808 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" containerName="manager" Jul 07 15:57:49.792585 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.792567 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:49.795605 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.795589 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 15:57:49.796957 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.796935 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wk7g6\"" Jul 07 15:57:49.800192 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.797018 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 15:57:49.807002 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.806977 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796b9b6788-85hkb"] Jul 07 15:57:49.910326 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.910288 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm7fl\" (UniqueName: \"kubernetes.io/projected/0d13d56b-4b90-4f93-8331-9da4702c85b2-kube-api-access-nm7fl\") pod \"maas-controller-796b9b6788-85hkb\" (UID: \"0d13d56b-4b90-4f93-8331-9da4702c85b2\") " pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:49.910502 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:49.910382 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d13d56b-4b90-4f93-8331-9da4702c85b2-webhook-cert\") pod \"maas-controller-796b9b6788-85hkb\" (UID: \"0d13d56b-4b90-4f93-8331-9da4702c85b2\") " pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.011313 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.011273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d13d56b-4b90-4f93-8331-9da4702c85b2-webhook-cert\") pod \"maas-controller-796b9b6788-85hkb\" (UID: \"0d13d56b-4b90-4f93-8331-9da4702c85b2\") " pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.011500 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.011361 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm7fl\" (UniqueName: \"kubernetes.io/projected/0d13d56b-4b90-4f93-8331-9da4702c85b2-kube-api-access-nm7fl\") pod \"maas-controller-796b9b6788-85hkb\" (UID: \"0d13d56b-4b90-4f93-8331-9da4702c85b2\") " pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.013848 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.013823 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d13d56b-4b90-4f93-8331-9da4702c85b2-webhook-cert\") pod \"maas-controller-796b9b6788-85hkb\" (UID: \"0d13d56b-4b90-4f93-8331-9da4702c85b2\") " pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.021796 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.021770 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm7fl\" (UniqueName: \"kubernetes.io/projected/0d13d56b-4b90-4f93-8331-9da4702c85b2-kube-api-access-nm7fl\") pod \"maas-controller-796b9b6788-85hkb\" (UID: \"0d13d56b-4b90-4f93-8331-9da4702c85b2\") " pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.032049 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.032023 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02" path="/var/lib/kubelet/pods/9ebcca26-4da4-49cc-9c0c-6eadd9fd5c02/volumes" Jul 07 15:57:50.108906 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.108821 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.246105 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.246080 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796b9b6788-85hkb"] Jul 07 15:57:50.247999 ip-10-0-136-171 kubenswrapper[2585]: W0707 15:57:50.247972 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d13d56b_4b90_4f93_8331_9da4702c85b2.slice/crio-6888a11f95a9f4dfae59de09d6885ba5dfbf3fc69aac5781b2b3f74477933e52 WatchSource:0}: Error finding container 6888a11f95a9f4dfae59de09d6885ba5dfbf3fc69aac5781b2b3f74477933e52: Status 404 returned error can't find the container with id 6888a11f95a9f4dfae59de09d6885ba5dfbf3fc69aac5781b2b3f74477933e52 Jul 07 15:57:50.951237 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.951122 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796b9b6788-85hkb" event={"ID":"0d13d56b-4b90-4f93-8331-9da4702c85b2","Type":"ContainerStarted","Data":"c36d28d755bda8aaff42a23acf3f61523a5c5ee6061195a4a2ed787a1fbba372"} Jul 07 15:57:50.951237 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.951199 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796b9b6788-85hkb" event={"ID":"0d13d56b-4b90-4f93-8331-9da4702c85b2","Type":"ContainerStarted","Data":"6888a11f95a9f4dfae59de09d6885ba5dfbf3fc69aac5781b2b3f74477933e52"} Jul 07 15:57:50.951762 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.951241 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 15:57:50.973049 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:57:50.972998 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-796b9b6788-85hkb" podStartSLOduration=1.623433535 podStartE2EDuration="1.972981301s" totalImagesPullingTime="349.547766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:57:50.970549518 +0000 UTC m=+805.462267982" watchObservedRunningTime="2026-07-07 15:57:50.972981301 +0000 UTC m=+805.464699764" Jul 07 15:58:01.962010 ip-10-0-136-171 kubenswrapper[2585]: I0707 15:58:01.961978 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-796b9b6788-85hkb" Jul 07 16:00:00.158777 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.158739 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn"] Jul 07 16:00:00.161231 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.161215 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:00:00.185132 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.185100 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn"] Jul 07 16:00:00.290097 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.290048 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5fb\" (UniqueName: \"kubernetes.io/projected/d9eb3f9a-2244-44a3-a6ce-acb344306b2d-kube-api-access-5x5fb\") pod \"maas-api-key-cleanup-29724000-6d2dn\" (UID: \"d9eb3f9a-2244-44a3-a6ce-acb344306b2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:00:00.391471 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.391433 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5fb\" (UniqueName: \"kubernetes.io/projected/d9eb3f9a-2244-44a3-a6ce-acb344306b2d-kube-api-access-5x5fb\") pod \"maas-api-key-cleanup-29724000-6d2dn\" (UID: \"d9eb3f9a-2244-44a3-a6ce-acb344306b2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:00:00.401977 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.401950 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5fb\" (UniqueName: \"kubernetes.io/projected/d9eb3f9a-2244-44a3-a6ce-acb344306b2d-kube-api-access-5x5fb\") pod \"maas-api-key-cleanup-29724000-6d2dn\" (UID: \"d9eb3f9a-2244-44a3-a6ce-acb344306b2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:00:00.471671 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.471583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:00:00.607546 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:00.607512 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn"] Jul 07 16:00:00.609829 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:00:00.609800 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9eb3f9a_2244_44a3_a6ce_acb344306b2d.slice/crio-1707f2e5c5ba4c992c084ba0050542978eed4670d1ddc75c730be84e7e4b77a5 WatchSource:0}: Error finding container 1707f2e5c5ba4c992c084ba0050542978eed4670d1ddc75c730be84e7e4b77a5: Status 404 returned error can't find the container with id 1707f2e5c5ba4c992c084ba0050542978eed4670d1ddc75c730be84e7e4b77a5 Jul 07 16:00:01.574659 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:01.574624 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" event={"ID":"d9eb3f9a-2244-44a3-a6ce-acb344306b2d","Type":"ContainerStarted","Data":"1707f2e5c5ba4c992c084ba0050542978eed4670d1ddc75c730be84e7e4b77a5"} Jul 07 16:00:06.602434 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:06.602400 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" event={"ID":"d9eb3f9a-2244-44a3-a6ce-acb344306b2d","Type":"ContainerStarted","Data":"2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86"} Jul 07 16:00:06.620409 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:00:06.620353 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" podStartSLOduration=1.530905103 podStartE2EDuration="6.620335234s" totalImagesPullingTime="5.089430131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:00:06.619847141 +0000 UTC m=+941.111565605" watchObservedRunningTime="2026-07-07 16:00:06.620335234 +0000 UTC m=+941.112053696" Jul 07 16:01:00.013009 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:00.012917 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn"] Jul 07 16:01:00.013560 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:00.013213 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" podUID="d9eb3f9a-2244-44a3-a6ce-acb344306b2d" containerName="cleanup" containerID="cri-o://2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86" gracePeriod=30 Jul 07 16:01:30.189232 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:30.189203 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:01:30.248292 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:30.248261 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x5fb\" (UniqueName: \"kubernetes.io/projected/d9eb3f9a-2244-44a3-a6ce-acb344306b2d-kube-api-access-5x5fb\") pod \"d9eb3f9a-2244-44a3-a6ce-acb344306b2d\" (UID: \"d9eb3f9a-2244-44a3-a6ce-acb344306b2d\") " Jul 07 16:01:30.250547 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:30.250519 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9eb3f9a-2244-44a3-a6ce-acb344306b2d-kube-api-access-5x5fb" (OuterVolumeSpecName: "kube-api-access-5x5fb") pod "d9eb3f9a-2244-44a3-a6ce-acb344306b2d" (UID: "d9eb3f9a-2244-44a3-a6ce-acb344306b2d"). InnerVolumeSpecName "kube-api-access-5x5fb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:01:30.349239 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:30.349130 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x5fb\" (UniqueName: \"kubernetes.io/projected/d9eb3f9a-2244-44a3-a6ce-acb344306b2d-kube-api-access-5x5fb\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:01:31.020615 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.020578 2585 generic.go:356] "Generic (PLEG): container finished" podID="d9eb3f9a-2244-44a3-a6ce-acb344306b2d" containerID="2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86" exitCode=137 Jul 07 16:01:31.020816 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.020626 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" Jul 07 16:01:31.020816 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.020662 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" event={"ID":"d9eb3f9a-2244-44a3-a6ce-acb344306b2d","Type":"ContainerDied","Data":"2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86"} Jul 07 16:01:31.020816 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.020717 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn" event={"ID":"d9eb3f9a-2244-44a3-a6ce-acb344306b2d","Type":"ContainerDied","Data":"1707f2e5c5ba4c992c084ba0050542978eed4670d1ddc75c730be84e7e4b77a5"} Jul 07 16:01:31.020816 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.020740 2585 scope.go:117] "RemoveContainer" containerID="2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86" Jul 07 16:01:31.036256 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.036235 2585 scope.go:117] "RemoveContainer" containerID="2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86" Jul 07 16:01:31.036526 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:01:31.036507 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86\": container with ID starting with 2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86 not found: ID does not exist" containerID="2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86" Jul 07 16:01:31.036590 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.036539 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86"} err="failed to get container status \"2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86\": rpc error: code = NotFound desc = could not find container \"2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86\": container with ID starting with 2d387d0b3d8d40f36830c2b3c98c572b880a33864f76848a42683cbe41b2fa86 not found: ID does not exist" Jul 07 16:01:31.047272 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.047249 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn"] Jul 07 16:01:31.051501 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:31.051473 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724000-6d2dn"] Jul 07 16:01:32.031220 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:01:32.031188 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9eb3f9a-2244-44a3-a6ce-acb344306b2d" path="/var/lib/kubelet/pods/d9eb3f9a-2244-44a3-a6ce-acb344306b2d/volumes" Jul 07 16:09:40.787281 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.787240 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw"] Jul 07 16:09:40.787902 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.787880 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9eb3f9a-2244-44a3-a6ce-acb344306b2d" containerName="cleanup" Jul 07 16:09:40.787948 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.787907 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9eb3f9a-2244-44a3-a6ce-acb344306b2d" containerName="cleanup" Jul 07 16:09:40.788026 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.788014 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9eb3f9a-2244-44a3-a6ce-acb344306b2d" containerName="cleanup" Jul 07 16:09:40.790928 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.790909 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:40.793605 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.793583 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-958428c6-serving-cert\"" Jul 07 16:09:40.802510 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.802478 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw"] Jul 07 16:09:40.972321 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.972274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96fc8181-6044-436c-b1e2-4d16020d5eda-openshift-service-ca\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:40.972321 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.972331 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4csr\" (UniqueName: \"kubernetes.io/projected/96fc8181-6044-436c-b1e2-4d16020d5eda-kube-api-access-w4csr\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:40.972581 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:40.972475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.073111 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.073012 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96fc8181-6044-436c-b1e2-4d16020d5eda-openshift-service-ca\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.073111 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.073074 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4csr\" (UniqueName: \"kubernetes.io/projected/96fc8181-6044-436c-b1e2-4d16020d5eda-kube-api-access-w4csr\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.073370 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.073203 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.073370 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:09:41.073350 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-serving-cert: secret "maas-api-e2e-ait-958428c6-serving-cert" not found Jul 07 16:09:41.073454 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:09:41.073441 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls podName:96fc8181-6044-436c-b1e2-4d16020d5eda nodeName:}" failed. No retries permitted until 2026-07-07 16:09:41.573417901 +0000 UTC m=+1516.065136342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls") pod "maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" (UID: "96fc8181-6044-436c-b1e2-4d16020d5eda") : secret "maas-api-e2e-ait-958428c6-serving-cert" not found Jul 07 16:09:41.073812 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.073794 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96fc8181-6044-436c-b1e2-4d16020d5eda-openshift-service-ca\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.082243 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.082216 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4csr\" (UniqueName: \"kubernetes.io/projected/96fc8181-6044-436c-b1e2-4d16020d5eda-kube-api-access-w4csr\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.578392 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.578349 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.580914 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.580892 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls\") pod \"maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.703711 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.703672 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:41.848760 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.848682 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw"] Jul 07 16:09:41.851201 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:09:41.851150 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fc8181_6044_436c_b1e2_4d16020d5eda.slice/crio-bcdab144f7996f98593fcc0d08ceb0dafeca9007a1409ff5c5d4f3cfc0adc4ee WatchSource:0}: Error finding container bcdab144f7996f98593fcc0d08ceb0dafeca9007a1409ff5c5d4f3cfc0adc4ee: Status 404 returned error can't find the container with id bcdab144f7996f98593fcc0d08ceb0dafeca9007a1409ff5c5d4f3cfc0adc4ee Jul 07 16:09:41.852547 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:41.852531 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:09:42.385899 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:42.385863 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" event={"ID":"96fc8181-6044-436c-b1e2-4d16020d5eda","Type":"ContainerStarted","Data":"bcdab144f7996f98593fcc0d08ceb0dafeca9007a1409ff5c5d4f3cfc0adc4ee"} Jul 07 16:09:43.393557 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:43.393517 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" event={"ID":"96fc8181-6044-436c-b1e2-4d16020d5eda","Type":"ContainerStarted","Data":"f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850"} Jul 07 16:09:43.393987 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:43.393584 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:43.413276 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:43.413222 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" podStartSLOduration=2.8568893429999997 podStartE2EDuration="3.413204729s" totalImagesPullingTime="556.315386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:09:43.411721587 +0000 UTC m=+1517.903440052" watchObservedRunningTime="2026-07-07 16:09:43.413204729 +0000 UTC m=+1517.904923192" Jul 07 16:09:46.041183 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.041134 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw"] Jul 07 16:09:46.041669 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.041383 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" podUID="96fc8181-6044-436c-b1e2-4d16020d5eda" containerName="maas-api" containerID="cri-o://f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850" gracePeriod=30 Jul 07 16:09:46.290278 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.290251 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:46.407610 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.407526 2585 generic.go:356] "Generic (PLEG): container finished" podID="96fc8181-6044-436c-b1e2-4d16020d5eda" containerID="f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850" exitCode=0 Jul 07 16:09:46.407610 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.407574 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" event={"ID":"96fc8181-6044-436c-b1e2-4d16020d5eda","Type":"ContainerDied","Data":"f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850"} Jul 07 16:09:46.407610 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.407611 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" event={"ID":"96fc8181-6044-436c-b1e2-4d16020d5eda","Type":"ContainerDied","Data":"bcdab144f7996f98593fcc0d08ceb0dafeca9007a1409ff5c5d4f3cfc0adc4ee"} Jul 07 16:09:46.407848 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.407631 2585 scope.go:117] "RemoveContainer" containerID="f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850" Jul 07 16:09:46.407848 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.407577 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw" Jul 07 16:09:46.416130 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.416106 2585 scope.go:117] "RemoveContainer" containerID="f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850" Jul 07 16:09:46.416424 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:09:46.416399 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850\": container with ID starting with f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850 not found: ID does not exist" containerID="f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850" Jul 07 16:09:46.416491 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.416429 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850"} err="failed to get container status \"f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850\": rpc error: code = NotFound desc = could not find container \"f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850\": container with ID starting with f2ad82e022364304a97fced74efc16325730373b2ab4b33118c899d254235850 not found: ID does not exist" Jul 07 16:09:46.425761 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.425741 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4csr\" (UniqueName: \"kubernetes.io/projected/96fc8181-6044-436c-b1e2-4d16020d5eda-kube-api-access-w4csr\") pod \"96fc8181-6044-436c-b1e2-4d16020d5eda\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " Jul 07 16:09:46.425883 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.425807 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls\") pod \"96fc8181-6044-436c-b1e2-4d16020d5eda\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " Jul 07 16:09:46.425883 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.425848 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96fc8181-6044-436c-b1e2-4d16020d5eda-openshift-service-ca\") pod \"96fc8181-6044-436c-b1e2-4d16020d5eda\" (UID: \"96fc8181-6044-436c-b1e2-4d16020d5eda\") " Jul 07 16:09:46.426215 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.426196 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96fc8181-6044-436c-b1e2-4d16020d5eda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96fc8181-6044-436c-b1e2-4d16020d5eda" (UID: "96fc8181-6044-436c-b1e2-4d16020d5eda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:09:46.427863 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.427839 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96fc8181-6044-436c-b1e2-4d16020d5eda-kube-api-access-w4csr" (OuterVolumeSpecName: "kube-api-access-w4csr") pod "96fc8181-6044-436c-b1e2-4d16020d5eda" (UID: "96fc8181-6044-436c-b1e2-4d16020d5eda"). InnerVolumeSpecName "kube-api-access-w4csr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:09:46.427988 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.427967 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96fc8181-6044-436c-b1e2-4d16020d5eda" (UID: "96fc8181-6044-436c-b1e2-4d16020d5eda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:09:46.526608 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.526573 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4csr\" (UniqueName: \"kubernetes.io/projected/96fc8181-6044-436c-b1e2-4d16020d5eda-kube-api-access-w4csr\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:09:46.526608 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.526606 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96fc8181-6044-436c-b1e2-4d16020d5eda-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:09:46.526814 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.526621 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96fc8181-6044-436c-b1e2-4d16020d5eda-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:09:46.729889 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.729855 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw"] Jul 07 16:09:46.736794 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:46.736765 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-958428c6-5d4fc88989-2bkvw"] Jul 07 16:09:48.031341 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:48.031306 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96fc8181-6044-436c-b1e2-4d16020d5eda" path="/var/lib/kubelet/pods/96fc8181-6044-436c-b1e2-4d16020d5eda/volumes" Jul 07 16:09:53.518448 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.518409 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq"] Jul 07 16:09:53.519090 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.519069 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96fc8181-6044-436c-b1e2-4d16020d5eda" containerName="maas-api" Jul 07 16:09:53.519137 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.519095 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="96fc8181-6044-436c-b1e2-4d16020d5eda" containerName="maas-api" Jul 07 16:09:53.519326 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.519310 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="96fc8181-6044-436c-b1e2-4d16020d5eda" containerName="maas-api" Jul 07 16:09:53.524030 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.524009 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.526541 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.526516 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f8a48a23-serving-cert\"" Jul 07 16:09:53.533879 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.533846 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq"] Jul 07 16:09:53.589870 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.589827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw957\" (UniqueName: \"kubernetes.io/projected/d33965fd-c2e1-430f-836a-92dc6c552c3d-kube-api-access-lw957\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.590056 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.589899 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.590056 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.589947 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33965fd-c2e1-430f-836a-92dc6c552c3d-openshift-service-ca\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.690825 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.690764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw957\" (UniqueName: \"kubernetes.io/projected/d33965fd-c2e1-430f-836a-92dc6c552c3d-kube-api-access-lw957\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.690825 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.690840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.691061 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.690876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33965fd-c2e1-430f-836a-92dc6c552c3d-openshift-service-ca\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.691061 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:09:53.690999 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-serving-cert: secret "maas-api-e2e-ait-f8a48a23-serving-cert" not found Jul 07 16:09:53.691131 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:09:53.691074 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls podName:d33965fd-c2e1-430f-836a-92dc6c552c3d nodeName:}" failed. No retries permitted until 2026-07-07 16:09:54.191049472 +0000 UTC m=+1528.682767914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls") pod "maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" (UID: "d33965fd-c2e1-430f-836a-92dc6c552c3d") : secret "maas-api-e2e-ait-f8a48a23-serving-cert" not found Jul 07 16:09:53.691560 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.691542 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33965fd-c2e1-430f-836a-92dc6c552c3d-openshift-service-ca\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:53.701795 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:53.701761 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw957\" (UniqueName: \"kubernetes.io/projected/d33965fd-c2e1-430f-836a-92dc6c552c3d-kube-api-access-lw957\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:54.196646 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:54.196594 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:54.199358 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:54.199319 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls\") pod \"maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:54.436951 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:54.436911 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:54.585320 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:54.585230 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq"] Jul 07 16:09:54.587824 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:09:54.587786 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33965fd_c2e1_430f_836a_92dc6c552c3d.slice/crio-d21f35d04779f2dc24ae0d33500421652c8f68cf75368ebefb106e0de4cb1acb WatchSource:0}: Error finding container d21f35d04779f2dc24ae0d33500421652c8f68cf75368ebefb106e0de4cb1acb: Status 404 returned error can't find the container with id d21f35d04779f2dc24ae0d33500421652c8f68cf75368ebefb106e0de4cb1acb Jul 07 16:09:55.457452 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:55.457356 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" event={"ID":"d33965fd-c2e1-430f-836a-92dc6c552c3d","Type":"ContainerStarted","Data":"a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a"} Jul 07 16:09:55.457452 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:55.457407 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" event={"ID":"d33965fd-c2e1-430f-836a-92dc6c552c3d","Type":"ContainerStarted","Data":"d21f35d04779f2dc24ae0d33500421652c8f68cf75368ebefb106e0de4cb1acb"} Jul 07 16:09:55.457695 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:55.457474 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:55.477266 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:55.477214 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" podStartSLOduration=2.015479752 podStartE2EDuration="2.477195366s" totalImagesPullingTime="461.715614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:09:55.475428773 +0000 UTC m=+1529.967147237" watchObservedRunningTime="2026-07-07 16:09:55.477195366 +0000 UTC m=+1529.968913829" Jul 07 16:09:58.880254 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:58.880222 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq"] Jul 07 16:09:58.880694 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:58.880534 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" podUID="d33965fd-c2e1-430f-836a-92dc6c552c3d" containerName="maas-api" containerID="cri-o://a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a" gracePeriod=30 Jul 07 16:09:59.127317 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.127286 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:59.246785 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.246746 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw957\" (UniqueName: \"kubernetes.io/projected/d33965fd-c2e1-430f-836a-92dc6c552c3d-kube-api-access-lw957\") pod \"d33965fd-c2e1-430f-836a-92dc6c552c3d\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " Jul 07 16:09:59.246945 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.246823 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls\") pod \"d33965fd-c2e1-430f-836a-92dc6c552c3d\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " Jul 07 16:09:59.246945 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.246851 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33965fd-c2e1-430f-836a-92dc6c552c3d-openshift-service-ca\") pod \"d33965fd-c2e1-430f-836a-92dc6c552c3d\" (UID: \"d33965fd-c2e1-430f-836a-92dc6c552c3d\") " Jul 07 16:09:59.247295 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.247269 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d33965fd-c2e1-430f-836a-92dc6c552c3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d33965fd-c2e1-430f-836a-92dc6c552c3d" (UID: "d33965fd-c2e1-430f-836a-92dc6c552c3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:09:59.248990 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.248971 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d33965fd-c2e1-430f-836a-92dc6c552c3d" (UID: "d33965fd-c2e1-430f-836a-92dc6c552c3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:09:59.249131 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.249115 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33965fd-c2e1-430f-836a-92dc6c552c3d-kube-api-access-lw957" (OuterVolumeSpecName: "kube-api-access-lw957") pod "d33965fd-c2e1-430f-836a-92dc6c552c3d" (UID: "d33965fd-c2e1-430f-836a-92dc6c552c3d"). InnerVolumeSpecName "kube-api-access-lw957". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:09:59.347928 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.347893 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33965fd-c2e1-430f-836a-92dc6c552c3d-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:09:59.347928 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.347924 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33965fd-c2e1-430f-836a-92dc6c552c3d-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:09:59.347928 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.347934 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw957\" (UniqueName: \"kubernetes.io/projected/d33965fd-c2e1-430f-836a-92dc6c552c3d-kube-api-access-lw957\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:09:59.478938 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.478901 2585 generic.go:356] "Generic (PLEG): container finished" podID="d33965fd-c2e1-430f-836a-92dc6c552c3d" containerID="a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a" exitCode=0 Jul 07 16:09:59.479116 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.478953 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" Jul 07 16:09:59.479116 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.478970 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" event={"ID":"d33965fd-c2e1-430f-836a-92dc6c552c3d","Type":"ContainerDied","Data":"a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a"} Jul 07 16:09:59.479116 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.479015 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq" event={"ID":"d33965fd-c2e1-430f-836a-92dc6c552c3d","Type":"ContainerDied","Data":"d21f35d04779f2dc24ae0d33500421652c8f68cf75368ebefb106e0de4cb1acb"} Jul 07 16:09:59.479116 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.479036 2585 scope.go:117] "RemoveContainer" containerID="a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a" Jul 07 16:09:59.488575 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.488516 2585 scope.go:117] "RemoveContainer" containerID="a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a" Jul 07 16:09:59.488861 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:09:59.488840 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a\": container with ID starting with a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a not found: ID does not exist" containerID="a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a" Jul 07 16:09:59.488922 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.488870 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a"} err="failed to get container status \"a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a\": rpc error: code = NotFound desc = could not find container \"a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a\": container with ID starting with a05c7c5fcd40e22d043713ed631a77e8cf82b9a932fce9e88c552c0b7d19a26a not found: ID does not exist" Jul 07 16:09:59.500781 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.500688 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq"] Jul 07 16:09:59.504501 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:09:59.504473 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f8a48a23-d8bb564cf-kjbtq"] Jul 07 16:10:00.036557 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:00.036523 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d33965fd-c2e1-430f-836a-92dc6c552c3d" path="/var/lib/kubelet/pods/d33965fd-c2e1-430f-836a-92dc6c552c3d/volumes" Jul 07 16:10:12.301225 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.301186 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824"] Jul 07 16:10:12.301870 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.301844 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33965fd-c2e1-430f-836a-92dc6c552c3d" containerName="maas-api" Jul 07 16:10:12.302024 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.301902 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33965fd-c2e1-430f-836a-92dc6c552c3d" containerName="maas-api" Jul 07 16:10:12.302106 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.302040 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d33965fd-c2e1-430f-836a-92dc6c552c3d" containerName="maas-api" Jul 07 16:10:12.305291 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.305264 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.307935 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.307912 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ce576197-serving-cert\"" Jul 07 16:10:12.318399 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.318369 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824"] Jul 07 16:10:12.380111 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.380071 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-maas-api-tls\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.380321 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.380241 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jfx6\" (UniqueName: \"kubernetes.io/projected/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-kube-api-access-9jfx6\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.380321 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.380287 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-openshift-service-ca\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.480750 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.480710 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jfx6\" (UniqueName: \"kubernetes.io/projected/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-kube-api-access-9jfx6\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.480750 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.480752 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-openshift-service-ca\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.480971 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.480781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-maas-api-tls\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.481464 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.481441 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-openshift-service-ca\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.483546 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.483519 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-maas-api-tls\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.489868 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.489844 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jfx6\" (UniqueName: \"kubernetes.io/projected/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-kube-api-access-9jfx6\") pod \"maas-api-e2e-derive-ce576197-97bd5c4-2z824\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.616521 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.616432 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:12.761846 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:12.761810 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824"] Jul 07 16:10:12.771421 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:10:12.771391 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f5ab2d_6ad1_4dab_9ffe_b609798c2050.slice/crio-d49f8e20b1a1bdfd401eba51f841fc1d7f7d8cab0cc2676d72b43b79df66a2fb WatchSource:0}: Error finding container d49f8e20b1a1bdfd401eba51f841fc1d7f7d8cab0cc2676d72b43b79df66a2fb: Status 404 returned error can't find the container with id d49f8e20b1a1bdfd401eba51f841fc1d7f7d8cab0cc2676d72b43b79df66a2fb Jul 07 16:10:13.552779 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:13.552742 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" event={"ID":"70f5ab2d-6ad1-4dab-9ffe-b609798c2050","Type":"ContainerStarted","Data":"35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2"} Jul 07 16:10:13.553190 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:13.552800 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:13.553190 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:13.552818 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" event={"ID":"70f5ab2d-6ad1-4dab-9ffe-b609798c2050","Type":"ContainerStarted","Data":"d49f8e20b1a1bdfd401eba51f841fc1d7f7d8cab0cc2676d72b43b79df66a2fb"} Jul 07 16:10:13.573464 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:13.573407 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" podStartSLOduration=1.20455885 podStartE2EDuration="1.573391648s" totalImagesPullingTime="368.832798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:10:13.571308342 +0000 UTC m=+1548.063026819" watchObservedRunningTime="2026-07-07 16:10:13.573391648 +0000 UTC m=+1548.065110110" Jul 07 16:10:16.898072 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:16.898032 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824"] Jul 07 16:10:16.898530 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:16.898291 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" podUID="70f5ab2d-6ad1-4dab-9ffe-b609798c2050" containerName="maas-api" containerID="cri-o://35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2" gracePeriod=30 Jul 07 16:10:17.143605 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.143580 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:17.231653 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.231563 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jfx6\" (UniqueName: \"kubernetes.io/projected/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-kube-api-access-9jfx6\") pod \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " Jul 07 16:10:17.231653 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.231599 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-maas-api-tls\") pod \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " Jul 07 16:10:17.231877 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.231680 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-openshift-service-ca\") pod \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\" (UID: \"70f5ab2d-6ad1-4dab-9ffe-b609798c2050\") " Jul 07 16:10:17.232113 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.232085 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70f5ab2d-6ad1-4dab-9ffe-b609798c2050" (UID: "70f5ab2d-6ad1-4dab-9ffe-b609798c2050"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:10:17.233935 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.233903 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-kube-api-access-9jfx6" (OuterVolumeSpecName: "kube-api-access-9jfx6") pod "70f5ab2d-6ad1-4dab-9ffe-b609798c2050" (UID: "70f5ab2d-6ad1-4dab-9ffe-b609798c2050"). InnerVolumeSpecName "kube-api-access-9jfx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:17.234034 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.233957 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70f5ab2d-6ad1-4dab-9ffe-b609798c2050" (UID: "70f5ab2d-6ad1-4dab-9ffe-b609798c2050"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:10:17.332656 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.332614 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jfx6\" (UniqueName: \"kubernetes.io/projected/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-kube-api-access-9jfx6\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:10:17.332656 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.332653 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:10:17.332881 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.332667 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f5ab2d-6ad1-4dab-9ffe-b609798c2050-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:10:17.574564 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.574527 2585 generic.go:356] "Generic (PLEG): container finished" podID="70f5ab2d-6ad1-4dab-9ffe-b609798c2050" containerID="35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2" exitCode=0 Jul 07 16:10:17.574838 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.574589 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" Jul 07 16:10:17.574838 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.574606 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" event={"ID":"70f5ab2d-6ad1-4dab-9ffe-b609798c2050","Type":"ContainerDied","Data":"35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2"} Jul 07 16:10:17.574838 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.574689 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824" event={"ID":"70f5ab2d-6ad1-4dab-9ffe-b609798c2050","Type":"ContainerDied","Data":"d49f8e20b1a1bdfd401eba51f841fc1d7f7d8cab0cc2676d72b43b79df66a2fb"} Jul 07 16:10:17.574838 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.574711 2585 scope.go:117] "RemoveContainer" containerID="35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2" Jul 07 16:10:17.584539 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.584516 2585 scope.go:117] "RemoveContainer" containerID="35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2" Jul 07 16:10:17.584854 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:10:17.584835 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2\": container with ID starting with 35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2 not found: ID does not exist" containerID="35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2" Jul 07 16:10:17.584920 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.584867 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2"} err="failed to get container status \"35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2\": rpc error: code = NotFound desc = could not find container \"35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2\": container with ID starting with 35ab04190e99ebe9d2c356aba2331eb7d2c418b96736c65af5e8da2b2582e2f2 not found: ID does not exist" Jul 07 16:10:17.596617 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.596583 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824"] Jul 07 16:10:17.602320 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:17.602296 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce576197-97bd5c4-2z824"] Jul 07 16:10:18.031493 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:10:18.031463 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70f5ab2d-6ad1-4dab-9ffe-b609798c2050" path="/var/lib/kubelet/pods/70f5ab2d-6ad1-4dab-9ffe-b609798c2050/volumes" Jul 07 16:12:03.754422 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.754381 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg"] Jul 07 16:12:03.755050 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.755033 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f5ab2d-6ad1-4dab-9ffe-b609798c2050" containerName="maas-api" Jul 07 16:12:03.755095 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.755053 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f5ab2d-6ad1-4dab-9ffe-b609798c2050" containerName="maas-api" Jul 07 16:12:03.755186 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.755148 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f5ab2d-6ad1-4dab-9ffe-b609798c2050" containerName="maas-api" Jul 07 16:12:03.758674 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.758654 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.761442 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.761417 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-109c0ad8-serving-cert\"" Jul 07 16:12:03.769477 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.769452 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg"] Jul 07 16:12:03.849568 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.849530 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ec59d8-cbbc-412d-ad14-b6b80e14935d-openshift-service-ca\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.849758 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.849585 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rsg\" (UniqueName: \"kubernetes.io/projected/40ec59d8-cbbc-412d-ad14-b6b80e14935d-kube-api-access-h7rsg\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.849758 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.849611 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ec59d8-cbbc-412d-ad14-b6b80e14935d-maas-api-tls\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.951075 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.951042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ec59d8-cbbc-412d-ad14-b6b80e14935d-openshift-service-ca\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.951075 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.951091 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rsg\" (UniqueName: \"kubernetes.io/projected/40ec59d8-cbbc-412d-ad14-b6b80e14935d-kube-api-access-h7rsg\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.951325 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.951123 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ec59d8-cbbc-412d-ad14-b6b80e14935d-maas-api-tls\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.951780 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.951748 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ec59d8-cbbc-412d-ad14-b6b80e14935d-openshift-service-ca\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.953867 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.953846 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ec59d8-cbbc-412d-ad14-b6b80e14935d-maas-api-tls\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:03.960432 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:03.960404 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rsg\" (UniqueName: \"kubernetes.io/projected/40ec59d8-cbbc-412d-ad14-b6b80e14935d-kube-api-access-h7rsg\") pod \"maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:04.071089 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:04.070998 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:04.222622 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:04.222592 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg"] Jul 07 16:12:04.225930 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:12:04.225899 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ec59d8_cbbc_412d_ad14_b6b80e14935d.slice/crio-91398f459b686baab705d66a7eeb8b529c90c1ce40d53c6a430a370f883d6696 WatchSource:0}: Error finding container 91398f459b686baab705d66a7eeb8b529c90c1ce40d53c6a430a370f883d6696: Status 404 returned error can't find the container with id 91398f459b686baab705d66a7eeb8b529c90c1ce40d53c6a430a370f883d6696 Jul 07 16:12:05.099504 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:05.099457 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" event={"ID":"40ec59d8-cbbc-412d-ad14-b6b80e14935d","Type":"ContainerStarted","Data":"ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996"} Jul 07 16:12:05.099903 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:05.099524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" event={"ID":"40ec59d8-cbbc-412d-ad14-b6b80e14935d","Type":"ContainerStarted","Data":"91398f459b686baab705d66a7eeb8b529c90c1ce40d53c6a430a370f883d6696"} Jul 07 16:12:05.099903 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:05.099619 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:05.119955 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:05.119898 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" podStartSLOduration=1.636078192 podStartE2EDuration="2.119879057s" totalImagesPullingTime="483.800865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:12:05.117571035 +0000 UTC m=+1659.609289497" watchObservedRunningTime="2026-07-07 16:12:05.119879057 +0000 UTC m=+1659.611597578" Jul 07 16:12:11.111503 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:11.111462 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:16.271592 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.271557 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg"] Jul 07 16:12:16.272029 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.271789 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" podUID="40ec59d8-cbbc-412d-ad14-b6b80e14935d" containerName="maas-api" containerID="cri-o://ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996" gracePeriod=30 Jul 07 16:12:16.527204 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.527112 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:16.678354 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.678321 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ec59d8-cbbc-412d-ad14-b6b80e14935d-openshift-service-ca\") pod \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " Jul 07 16:12:16.678549 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.678367 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ec59d8-cbbc-412d-ad14-b6b80e14935d-maas-api-tls\") pod \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " Jul 07 16:12:16.678549 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.678398 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7rsg\" (UniqueName: \"kubernetes.io/projected/40ec59d8-cbbc-412d-ad14-b6b80e14935d-kube-api-access-h7rsg\") pod \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\" (UID: \"40ec59d8-cbbc-412d-ad14-b6b80e14935d\") " Jul 07 16:12:16.678786 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.678756 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40ec59d8-cbbc-412d-ad14-b6b80e14935d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40ec59d8-cbbc-412d-ad14-b6b80e14935d" (UID: "40ec59d8-cbbc-412d-ad14-b6b80e14935d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:12:16.680673 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.680646 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40ec59d8-cbbc-412d-ad14-b6b80e14935d-kube-api-access-h7rsg" (OuterVolumeSpecName: "kube-api-access-h7rsg") pod "40ec59d8-cbbc-412d-ad14-b6b80e14935d" (UID: "40ec59d8-cbbc-412d-ad14-b6b80e14935d"). InnerVolumeSpecName "kube-api-access-h7rsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:12:16.680744 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.680683 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40ec59d8-cbbc-412d-ad14-b6b80e14935d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40ec59d8-cbbc-412d-ad14-b6b80e14935d" (UID: "40ec59d8-cbbc-412d-ad14-b6b80e14935d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:12:16.779569 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.779457 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ec59d8-cbbc-412d-ad14-b6b80e14935d-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:12:16.779569 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.779509 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ec59d8-cbbc-412d-ad14-b6b80e14935d-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:12:16.779569 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:16.779521 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7rsg\" (UniqueName: \"kubernetes.io/projected/40ec59d8-cbbc-412d-ad14-b6b80e14935d-kube-api-access-h7rsg\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:12:17.161250 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.161139 2585 generic.go:356] "Generic (PLEG): container finished" podID="40ec59d8-cbbc-412d-ad14-b6b80e14935d" containerID="ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996" exitCode=0 Jul 07 16:12:17.161250 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.161219 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" event={"ID":"40ec59d8-cbbc-412d-ad14-b6b80e14935d","Type":"ContainerDied","Data":"ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996"} Jul 07 16:12:17.161450 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.161260 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" event={"ID":"40ec59d8-cbbc-412d-ad14-b6b80e14935d","Type":"ContainerDied","Data":"91398f459b686baab705d66a7eeb8b529c90c1ce40d53c6a430a370f883d6696"} Jul 07 16:12:17.161450 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.161281 2585 scope.go:117] "RemoveContainer" containerID="ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996" Jul 07 16:12:17.161450 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.161221 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg" Jul 07 16:12:17.177504 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.177476 2585 scope.go:117] "RemoveContainer" containerID="ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996" Jul 07 16:12:17.177862 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:12:17.177836 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996\": container with ID starting with ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996 not found: ID does not exist" containerID="ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996" Jul 07 16:12:17.177903 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.177876 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996"} err="failed to get container status \"ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996\": rpc error: code = NotFound desc = could not find container \"ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996\": container with ID starting with ad8c528801bfaa0115be45b220f5126a0c3941cdb9632f447aa6de865042e996 not found: ID does not exist" Jul 07 16:12:17.187442 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.187409 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg"] Jul 07 16:12:17.191056 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:17.191026 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-109c0ad8-7774cbc7c-s85lg"] Jul 07 16:12:18.031176 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:18.031122 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40ec59d8-cbbc-412d-ad14-b6b80e14935d" path="/var/lib/kubelet/pods/40ec59d8-cbbc-412d-ad14-b6b80e14935d/volumes" Jul 07 16:12:46.766821 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.766737 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz"] Jul 07 16:12:46.767424 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.767359 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40ec59d8-cbbc-412d-ad14-b6b80e14935d" containerName="maas-api" Jul 07 16:12:46.767424 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.767381 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="40ec59d8-cbbc-412d-ad14-b6b80e14935d" containerName="maas-api" Jul 07 16:12:46.767554 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.767528 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="40ec59d8-cbbc-412d-ad14-b6b80e14935d" containerName="maas-api" Jul 07 16:12:46.772292 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.772270 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.774804 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.774780 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cadb2345-serving-cert\"" Jul 07 16:12:46.782042 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.782007 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz"] Jul 07 16:12:46.869905 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.869854 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c73d76e-8651-4954-9d36-91f1b358603a-openshift-service-ca\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.870110 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.869946 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-926gt\" (UniqueName: \"kubernetes.io/projected/3c73d76e-8651-4954-9d36-91f1b358603a-kube-api-access-926gt\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.870110 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.869976 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c73d76e-8651-4954-9d36-91f1b358603a-maas-api-tls\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.970677 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.970634 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-926gt\" (UniqueName: \"kubernetes.io/projected/3c73d76e-8651-4954-9d36-91f1b358603a-kube-api-access-926gt\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.970888 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.970684 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c73d76e-8651-4954-9d36-91f1b358603a-maas-api-tls\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.970888 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.970783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c73d76e-8651-4954-9d36-91f1b358603a-openshift-service-ca\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.971516 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.971493 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c73d76e-8651-4954-9d36-91f1b358603a-openshift-service-ca\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.973400 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.973378 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c73d76e-8651-4954-9d36-91f1b358603a-maas-api-tls\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:46.978898 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:46.978873 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-926gt\" (UniqueName: \"kubernetes.io/projected/3c73d76e-8651-4954-9d36-91f1b358603a-kube-api-access-926gt\") pod \"maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:47.084310 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:47.084210 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:47.236812 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:47.236760 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz"] Jul 07 16:12:47.239286 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:12:47.239228 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c73d76e_8651_4954_9d36_91f1b358603a.slice/crio-840d9effb71a485690a379e9c48ddfc814fbf2d233fad54d6bef1c61b00c1225 WatchSource:0}: Error finding container 840d9effb71a485690a379e9c48ddfc814fbf2d233fad54d6bef1c61b00c1225: Status 404 returned error can't find the container with id 840d9effb71a485690a379e9c48ddfc814fbf2d233fad54d6bef1c61b00c1225 Jul 07 16:12:47.317111 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:47.317069 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" event={"ID":"3c73d76e-8651-4954-9d36-91f1b358603a","Type":"ContainerStarted","Data":"840d9effb71a485690a379e9c48ddfc814fbf2d233fad54d6bef1c61b00c1225"} Jul 07 16:12:48.324099 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:48.324066 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" event={"ID":"3c73d76e-8651-4954-9d36-91f1b358603a","Type":"ContainerStarted","Data":"d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2"} Jul 07 16:12:48.324606 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:48.324216 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:12:48.345103 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:48.345034 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" podStartSLOduration=1.948609217 podStartE2EDuration="2.345008286s" totalImagesPullingTime="396.399069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:12:48.34228442 +0000 UTC m=+1702.834002884" watchObservedRunningTime="2026-07-07 16:12:48.345008286 +0000 UTC m=+1702.836726750" Jul 07 16:12:54.335265 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:12:54.335233 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:13:39.755237 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:39.755205 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz"] Jul 07 16:13:39.755775 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:39.755403 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" podUID="3c73d76e-8651-4954-9d36-91f1b358603a" containerName="maas-api" containerID="cri-o://d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2" gracePeriod=30 Jul 07 16:13:40.004282 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.004253 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:13:40.090748 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.090664 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-926gt\" (UniqueName: \"kubernetes.io/projected/3c73d76e-8651-4954-9d36-91f1b358603a-kube-api-access-926gt\") pod \"3c73d76e-8651-4954-9d36-91f1b358603a\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " Jul 07 16:13:40.090748 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.090709 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c73d76e-8651-4954-9d36-91f1b358603a-openshift-service-ca\") pod \"3c73d76e-8651-4954-9d36-91f1b358603a\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " Jul 07 16:13:40.090953 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.090809 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c73d76e-8651-4954-9d36-91f1b358603a-maas-api-tls\") pod \"3c73d76e-8651-4954-9d36-91f1b358603a\" (UID: \"3c73d76e-8651-4954-9d36-91f1b358603a\") " Jul 07 16:13:40.091148 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.091116 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c73d76e-8651-4954-9d36-91f1b358603a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c73d76e-8651-4954-9d36-91f1b358603a" (UID: "3c73d76e-8651-4954-9d36-91f1b358603a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:13:40.093008 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.092986 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c73d76e-8651-4954-9d36-91f1b358603a-kube-api-access-926gt" (OuterVolumeSpecName: "kube-api-access-926gt") pod "3c73d76e-8651-4954-9d36-91f1b358603a" (UID: "3c73d76e-8651-4954-9d36-91f1b358603a"). InnerVolumeSpecName "kube-api-access-926gt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:13:40.093008 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.092996 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c73d76e-8651-4954-9d36-91f1b358603a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c73d76e-8651-4954-9d36-91f1b358603a" (UID: "3c73d76e-8651-4954-9d36-91f1b358603a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:13:40.193289 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.193254 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c73d76e-8651-4954-9d36-91f1b358603a-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:13:40.193289 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.193287 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-926gt\" (UniqueName: \"kubernetes.io/projected/3c73d76e-8651-4954-9d36-91f1b358603a-kube-api-access-926gt\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:13:40.193508 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.193301 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c73d76e-8651-4954-9d36-91f1b358603a-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:13:40.587483 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.587445 2585 generic.go:356] "Generic (PLEG): container finished" podID="3c73d76e-8651-4954-9d36-91f1b358603a" containerID="d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2" exitCode=0 Jul 07 16:13:40.587657 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.587498 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" Jul 07 16:13:40.587657 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.587533 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" event={"ID":"3c73d76e-8651-4954-9d36-91f1b358603a","Type":"ContainerDied","Data":"d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2"} Jul 07 16:13:40.587657 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.587587 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz" event={"ID":"3c73d76e-8651-4954-9d36-91f1b358603a","Type":"ContainerDied","Data":"840d9effb71a485690a379e9c48ddfc814fbf2d233fad54d6bef1c61b00c1225"} Jul 07 16:13:40.587657 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.587608 2585 scope.go:117] "RemoveContainer" containerID="d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2" Jul 07 16:13:40.596767 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.596748 2585 scope.go:117] "RemoveContainer" containerID="d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2" Jul 07 16:13:40.597103 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:13:40.597082 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2\": container with ID starting with d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2 not found: ID does not exist" containerID="d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2" Jul 07 16:13:40.597243 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.597112 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2"} err="failed to get container status \"d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2\": rpc error: code = NotFound desc = could not find container \"d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2\": container with ID starting with d98b8f46d5ef1f803f60d1f398d17e91314831936b4424cce5486c99ca2659a2 not found: ID does not exist" Jul 07 16:13:40.609095 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.609064 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz"] Jul 07 16:13:40.612490 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:40.612467 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cadb2345-89d8d4dbf-5k5cz"] Jul 07 16:13:42.030830 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:13:42.030797 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c73d76e-8651-4954-9d36-91f1b358603a" path="/var/lib/kubelet/pods/3c73d76e-8651-4954-9d36-91f1b358603a/volumes" Jul 07 16:15:33.663991 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.663955 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb"] Jul 07 16:15:33.666518 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.664566 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c73d76e-8651-4954-9d36-91f1b358603a" containerName="maas-api" Jul 07 16:15:33.666518 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.664583 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c73d76e-8651-4954-9d36-91f1b358603a" containerName="maas-api" Jul 07 16:15:33.666518 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.664668 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c73d76e-8651-4954-9d36-91f1b358603a" containerName="maas-api" Jul 07 16:15:33.667729 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.667707 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.671770 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.671752 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-be8b55-serving-cert\"" Jul 07 16:15:33.681446 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.681426 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb"] Jul 07 16:15:33.799990 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.799952 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.799990 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.799991 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-maas-api-tls\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.800244 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.800139 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg77s\" (UniqueName: \"kubernetes.io/projected/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-kube-api-access-vg77s\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.901699 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.901658 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg77s\" (UniqueName: \"kubernetes.io/projected/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-kube-api-access-vg77s\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.901875 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.901717 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.901875 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.901739 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-maas-api-tls\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.902480 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.902459 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.904341 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.904324 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-maas-api-tls\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.914601 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.914543 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg77s\" (UniqueName: \"kubernetes.io/projected/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-kube-api-access-vg77s\") pod \"maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:33.978633 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:33.978605 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:34.123342 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:34.123173 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb"] Jul 07 16:15:34.125613 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:15:34.125582 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdbeacf_0400_4b5c_9b6f_cca19ee3a79c.slice/crio-deafd23f5b0c4ac1aab7d25ea841a9c3e8981e9f2ca7a91bdbc76b878c44cef8 WatchSource:0}: Error finding container deafd23f5b0c4ac1aab7d25ea841a9c3e8981e9f2ca7a91bdbc76b878c44cef8: Status 404 returned error can't find the container with id deafd23f5b0c4ac1aab7d25ea841a9c3e8981e9f2ca7a91bdbc76b878c44cef8 Jul 07 16:15:34.126957 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:34.126939 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:15:34.166498 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:34.166415 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" event={"ID":"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c","Type":"ContainerStarted","Data":"deafd23f5b0c4ac1aab7d25ea841a9c3e8981e9f2ca7a91bdbc76b878c44cef8"} Jul 07 16:15:35.172863 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:35.172825 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" event={"ID":"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c","Type":"ContainerStarted","Data":"516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42"} Jul 07 16:15:35.173293 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:35.172966 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:35.194453 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:35.194401 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" podStartSLOduration=1.611304301 podStartE2EDuration="2.19438512s" totalImagesPullingTime="583.080819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:35.192410302 +0000 UTC m=+1869.684128766" watchObservedRunningTime="2026-07-07 16:15:35.19438512 +0000 UTC m=+1869.686103582" Jul 07 16:15:41.183322 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.183282 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:15:41.946693 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.946662 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z"] Jul 07 16:15:41.950669 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.950645 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:41.953368 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.953347 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-76e259-serving-cert\"" Jul 07 16:15:41.967153 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.967125 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z"] Jul 07 16:15:41.974730 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.974692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvfzk\" (UniqueName: \"kubernetes.io/projected/63a337b3-2a8a-4442-a27c-15d7517e3bd9-kube-api-access-kvfzk\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:41.974888 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.974811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:41.974888 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:41.974868 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a337b3-2a8a-4442-a27c-15d7517e3bd9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.076226 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:42.076196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.076407 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:42.076240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a337b3-2a8a-4442-a27c-15d7517e3bd9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.076407 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:42.076298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvfzk\" (UniqueName: \"kubernetes.io/projected/63a337b3-2a8a-4442-a27c-15d7517e3bd9-kube-api-access-kvfzk\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.076407 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:15:42.076355 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-serving-cert: secret "maas-api-e2e-inf-b-76e259-serving-cert" not found Jul 07 16:15:42.076534 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:15:42.076427 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls podName:63a337b3-2a8a-4442-a27c-15d7517e3bd9 nodeName:}" failed. No retries permitted until 2026-07-07 16:15:42.576406964 +0000 UTC m=+1877.068125410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls") pod "maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" (UID: "63a337b3-2a8a-4442-a27c-15d7517e3bd9") : secret "maas-api-e2e-inf-b-76e259-serving-cert" not found Jul 07 16:15:42.076902 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:42.076883 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a337b3-2a8a-4442-a27c-15d7517e3bd9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.092965 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:42.092939 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvfzk\" (UniqueName: \"kubernetes.io/projected/63a337b3-2a8a-4442-a27c-15d7517e3bd9-kube-api-access-kvfzk\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.581232 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:42.581192 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:42.581638 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:15:42.581329 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-serving-cert: secret "maas-api-e2e-inf-b-76e259-serving-cert" not found Jul 07 16:15:42.581638 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:15:42.581400 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls podName:63a337b3-2a8a-4442-a27c-15d7517e3bd9 nodeName:}" failed. No retries permitted until 2026-07-07 16:15:43.581383711 +0000 UTC m=+1878.073102151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls") pod "maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" (UID: "63a337b3-2a8a-4442-a27c-15d7517e3bd9") : secret "maas-api-e2e-inf-b-76e259-serving-cert" not found Jul 07 16:15:43.590094 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:43.590057 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:43.592751 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:43.592724 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:43.762516 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:43.762471 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:43.896134 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:43.896108 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z"] Jul 07 16:15:43.898219 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:15:43.898188 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a337b3_2a8a_4442_a27c_15d7517e3bd9.slice/crio-0022bce4a681f5171ce57d3af7b167da3818e02b8417d95e7725dce8aaf80cb2 WatchSource:0}: Error finding container 0022bce4a681f5171ce57d3af7b167da3818e02b8417d95e7725dce8aaf80cb2: Status 404 returned error can't find the container with id 0022bce4a681f5171ce57d3af7b167da3818e02b8417d95e7725dce8aaf80cb2 Jul 07 16:15:44.217179 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:44.217078 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" event={"ID":"63a337b3-2a8a-4442-a27c-15d7517e3bd9","Type":"ContainerStarted","Data":"0022bce4a681f5171ce57d3af7b167da3818e02b8417d95e7725dce8aaf80cb2"} Jul 07 16:15:45.223334 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:45.223298 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" event={"ID":"63a337b3-2a8a-4442-a27c-15d7517e3bd9","Type":"ContainerStarted","Data":"f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74"} Jul 07 16:15:45.223770 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:45.223422 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:51.233664 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:51.233581 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:15:51.256607 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:15:51.256552 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" podStartSLOduration=9.922632605 podStartE2EDuration="10.256534588s" totalImagesPullingTime="333.901983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:45.253586569 +0000 UTC m=+1879.745305030" watchObservedRunningTime="2026-07-07 16:15:51.256534588 +0000 UTC m=+1885.748253051" Jul 07 16:16:36.435190 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.435145 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb"] Jul 07 16:16:36.435740 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.435369 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" podUID="0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" containerName="maas-api" containerID="cri-o://516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42" gracePeriod=30 Jul 07 16:16:36.692705 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.692632 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:16:36.814223 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.814187 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg77s\" (UniqueName: \"kubernetes.io/projected/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-kube-api-access-vg77s\") pod \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " Jul 07 16:16:36.814439 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.814244 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-openshift-service-ca\") pod \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " Jul 07 16:16:36.814439 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.814266 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-maas-api-tls\") pod \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\" (UID: \"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c\") " Jul 07 16:16:36.814660 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.814635 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" (UID: "0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:16:36.816410 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.816388 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-kube-api-access-vg77s" (OuterVolumeSpecName: "kube-api-access-vg77s") pod "0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" (UID: "0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c"). InnerVolumeSpecName "kube-api-access-vg77s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:16:36.816490 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.816461 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" (UID: "0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:16:36.915359 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.915308 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg77s\" (UniqueName: \"kubernetes.io/projected/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-kube-api-access-vg77s\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:16:36.915359 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.915348 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:16:36.915359 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:36.915358 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:16:37.487011 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.486969 2585 generic.go:356] "Generic (PLEG): container finished" podID="0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" containerID="516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42" exitCode=0 Jul 07 16:16:37.487517 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.487019 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" Jul 07 16:16:37.487517 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.487058 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" event={"ID":"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c","Type":"ContainerDied","Data":"516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42"} Jul 07 16:16:37.487517 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.487110 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb" event={"ID":"0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c","Type":"ContainerDied","Data":"deafd23f5b0c4ac1aab7d25ea841a9c3e8981e9f2ca7a91bdbc76b878c44cef8"} Jul 07 16:16:37.487517 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.487132 2585 scope.go:117] "RemoveContainer" containerID="516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42" Jul 07 16:16:37.496207 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.496187 2585 scope.go:117] "RemoveContainer" containerID="516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42" Jul 07 16:16:37.496501 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:16:37.496481 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42\": container with ID starting with 516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42 not found: ID does not exist" containerID="516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42" Jul 07 16:16:37.496552 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.496512 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42"} err="failed to get container status \"516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42\": rpc error: code = NotFound desc = could not find container \"516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42\": container with ID starting with 516a866c90099f3821cd7c9c714d082193e15360e76e8f38783c0de1313ede42 not found: ID does not exist" Jul 07 16:16:37.511825 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.511791 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb"] Jul 07 16:16:37.516937 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:37.516914 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be8b55-5b4bc78b9d-wpwmb"] Jul 07 16:16:38.031370 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:16:38.031333 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" path="/var/lib/kubelet/pods/0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c/volumes" Jul 07 16:17:20.939273 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:20.939188 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z"] Jul 07 16:17:20.939839 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:20.939444 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" podUID="63a337b3-2a8a-4442-a27c-15d7517e3bd9" containerName="maas-api" containerID="cri-o://f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74" gracePeriod=30 Jul 07 16:17:21.191897 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.191828 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:17:21.369320 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.369289 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") pod \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " Jul 07 16:17:21.369513 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.369348 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a337b3-2a8a-4442-a27c-15d7517e3bd9-openshift-service-ca\") pod \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " Jul 07 16:17:21.369513 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.369432 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvfzk\" (UniqueName: \"kubernetes.io/projected/63a337b3-2a8a-4442-a27c-15d7517e3bd9-kube-api-access-kvfzk\") pod \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\" (UID: \"63a337b3-2a8a-4442-a27c-15d7517e3bd9\") " Jul 07 16:17:21.369865 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.369726 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63a337b3-2a8a-4442-a27c-15d7517e3bd9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63a337b3-2a8a-4442-a27c-15d7517e3bd9" (UID: "63a337b3-2a8a-4442-a27c-15d7517e3bd9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:17:21.369865 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.369794 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a337b3-2a8a-4442-a27c-15d7517e3bd9-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:17:21.371726 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.371702 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63a337b3-2a8a-4442-a27c-15d7517e3bd9-kube-api-access-kvfzk" (OuterVolumeSpecName: "kube-api-access-kvfzk") pod "63a337b3-2a8a-4442-a27c-15d7517e3bd9" (UID: "63a337b3-2a8a-4442-a27c-15d7517e3bd9"). InnerVolumeSpecName "kube-api-access-kvfzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:17:21.371788 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.371719 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63a337b3-2a8a-4442-a27c-15d7517e3bd9" (UID: "63a337b3-2a8a-4442-a27c-15d7517e3bd9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:17:21.470912 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.470808 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a337b3-2a8a-4442-a27c-15d7517e3bd9-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:17:21.470912 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.470844 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvfzk\" (UniqueName: \"kubernetes.io/projected/63a337b3-2a8a-4442-a27c-15d7517e3bd9-kube-api-access-kvfzk\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:17:21.710419 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.710382 2585 generic.go:356] "Generic (PLEG): container finished" podID="63a337b3-2a8a-4442-a27c-15d7517e3bd9" containerID="f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74" exitCode=0 Jul 07 16:17:21.710605 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.710448 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" Jul 07 16:17:21.710605 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.710472 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" event={"ID":"63a337b3-2a8a-4442-a27c-15d7517e3bd9","Type":"ContainerDied","Data":"f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74"} Jul 07 16:17:21.710605 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.710528 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z" event={"ID":"63a337b3-2a8a-4442-a27c-15d7517e3bd9","Type":"ContainerDied","Data":"0022bce4a681f5171ce57d3af7b167da3818e02b8417d95e7725dce8aaf80cb2"} Jul 07 16:17:21.710605 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.710551 2585 scope.go:117] "RemoveContainer" containerID="f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74" Jul 07 16:17:21.720097 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.720066 2585 scope.go:117] "RemoveContainer" containerID="f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74" Jul 07 16:17:21.720430 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:17:21.720404 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74\": container with ID starting with f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74 not found: ID does not exist" containerID="f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74" Jul 07 16:17:21.720493 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.720442 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74"} err="failed to get container status \"f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74\": rpc error: code = NotFound desc = could not find container \"f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74\": container with ID starting with f33e86bb076c0e4c4ba6f0b2ae4e364b266318915d11e88a8ca0e15f53ea8c74 not found: ID does not exist" Jul 07 16:17:21.759280 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.759245 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z"] Jul 07 16:17:21.768518 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:21.768488 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76e259-bd6bc4688-vs52z"] Jul 07 16:17:22.033613 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:22.033534 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63a337b3-2a8a-4442-a27c-15d7517e3bd9" path="/var/lib/kubelet/pods/63a337b3-2a8a-4442-a27c-15d7517e3bd9/volumes" Jul 07 16:17:33.479238 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479201 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8"] Jul 07 16:17:33.479709 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479615 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" containerName="maas-api" Jul 07 16:17:33.479709 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479626 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" containerName="maas-api" Jul 07 16:17:33.479709 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479641 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63a337b3-2a8a-4442-a27c-15d7517e3bd9" containerName="maas-api" Jul 07 16:17:33.479709 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479647 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63a337b3-2a8a-4442-a27c-15d7517e3bd9" containerName="maas-api" Jul 07 16:17:33.479709 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479698 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63a337b3-2a8a-4442-a27c-15d7517e3bd9" containerName="maas-api" Jul 07 16:17:33.479709 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.479707 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bdbeacf-0400-4b5c-9b6f-cca19ee3a79c" containerName="maas-api" Jul 07 16:17:33.482742 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.482721 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.485934 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.485908 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1dcebd-serving-cert\"" Jul 07 16:17:33.498326 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.498292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8"] Jul 07 16:17:33.579268 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.579229 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.579457 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.579373 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4qtn\" (UniqueName: \"kubernetes.io/projected/e9b01360-28f2-45ce-a9e2-ab450ba87484-kube-api-access-v4qtn\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.579457 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.579413 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9b01360-28f2-45ce-a9e2-ab450ba87484-openshift-service-ca\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.679961 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.679922 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4qtn\" (UniqueName: \"kubernetes.io/projected/e9b01360-28f2-45ce-a9e2-ab450ba87484-kube-api-access-v4qtn\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.679961 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.679978 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9b01360-28f2-45ce-a9e2-ab450ba87484-openshift-service-ca\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.680306 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.680040 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.680306 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:17:33.680231 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-serving-cert: secret "maas-api-e2e-api-a-1dcebd-serving-cert" not found Jul 07 16:17:33.680306 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:17:33.680302 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls podName:e9b01360-28f2-45ce-a9e2-ab450ba87484 nodeName:}" failed. No retries permitted until 2026-07-07 16:17:34.180282487 +0000 UTC m=+1988.672000931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls") pod "maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" (UID: "e9b01360-28f2-45ce-a9e2-ab450ba87484") : secret "maas-api-e2e-api-a-1dcebd-serving-cert" not found Jul 07 16:17:33.680700 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.680678 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9b01360-28f2-45ce-a9e2-ab450ba87484-openshift-service-ca\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:33.699896 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:33.699864 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4qtn\" (UniqueName: \"kubernetes.io/projected/e9b01360-28f2-45ce-a9e2-ab450ba87484-kube-api-access-v4qtn\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:34.184811 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:34.184764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:34.187610 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:34.187573 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls\") pod \"maas-api-e2e-api-a-1dcebd-868d94999-hfgj8\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:34.393086 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:34.393035 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:34.547854 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:34.547824 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8"] Jul 07 16:17:34.550093 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:17:34.550057 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b01360_28f2_45ce_a9e2_ab450ba87484.slice/crio-0a4d248c8e1c51f85837ed32ac7386d97bc76363c1a83b39b1443d10fc6dc7b5 WatchSource:0}: Error finding container 0a4d248c8e1c51f85837ed32ac7386d97bc76363c1a83b39b1443d10fc6dc7b5: Status 404 returned error can't find the container with id 0a4d248c8e1c51f85837ed32ac7386d97bc76363c1a83b39b1443d10fc6dc7b5 Jul 07 16:17:34.776337 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:34.776298 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" event={"ID":"e9b01360-28f2-45ce-a9e2-ab450ba87484","Type":"ContainerStarted","Data":"0a4d248c8e1c51f85837ed32ac7386d97bc76363c1a83b39b1443d10fc6dc7b5"} Jul 07 16:17:35.783118 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:35.783079 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" event={"ID":"e9b01360-28f2-45ce-a9e2-ab450ba87484","Type":"ContainerStarted","Data":"5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374"} Jul 07 16:17:35.783534 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:35.783199 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:35.814092 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:35.814037 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" podStartSLOduration=2.365295815 podStartE2EDuration="2.814017191s" totalImagesPullingTime="448.721376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:17:35.812719404 +0000 UTC m=+1990.304437868" watchObservedRunningTime="2026-07-07 16:17:35.814017191 +0000 UTC m=+1990.305735655" Jul 07 16:17:41.665571 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.665524 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d"] Jul 07 16:17:41.669643 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.669619 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.673520 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.673496 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e60798-serving-cert\"" Jul 07 16:17:41.681002 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.680962 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d"] Jul 07 16:17:41.763219 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.763145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d1556-2467-4c0a-9d65-9dc650766e14-openshift-service-ca\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.763402 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.763296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxj5z\" (UniqueName: \"kubernetes.io/projected/a07d1556-2467-4c0a-9d65-9dc650766e14-kube-api-access-cxj5z\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.763402 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.763383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d1556-2467-4c0a-9d65-9dc650766e14-maas-api-tls\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.794373 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.794345 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:41.864056 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.864021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d1556-2467-4c0a-9d65-9dc650766e14-openshift-service-ca\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.864270 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.864107 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxj5z\" (UniqueName: \"kubernetes.io/projected/a07d1556-2467-4c0a-9d65-9dc650766e14-kube-api-access-cxj5z\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.864270 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.864206 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d1556-2467-4c0a-9d65-9dc650766e14-maas-api-tls\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.864814 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.864785 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d1556-2467-4c0a-9d65-9dc650766e14-openshift-service-ca\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.867017 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.866986 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d1556-2467-4c0a-9d65-9dc650766e14-maas-api-tls\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.877549 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.877513 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxj5z\" (UniqueName: \"kubernetes.io/projected/a07d1556-2467-4c0a-9d65-9dc650766e14-kube-api-access-cxj5z\") pod \"maas-api-e2e-api-b-e60798-6884bb458b-mkx8d\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:41.980921 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:41.980826 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:42.133729 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:42.133694 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d"] Jul 07 16:17:42.138490 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:17:42.138308 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07d1556_2467_4c0a_9d65_9dc650766e14.slice/crio-e9d385419ac89a720d8fe9b9abdf812a4cae4cda0c2add5384b1c1678e15cefd WatchSource:0}: Error finding container e9d385419ac89a720d8fe9b9abdf812a4cae4cda0c2add5384b1c1678e15cefd: Status 404 returned error can't find the container with id e9d385419ac89a720d8fe9b9abdf812a4cae4cda0c2add5384b1c1678e15cefd Jul 07 16:17:42.820030 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:42.819926 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" event={"ID":"a07d1556-2467-4c0a-9d65-9dc650766e14","Type":"ContainerStarted","Data":"87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1"} Jul 07 16:17:42.820030 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:42.819995 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" event={"ID":"a07d1556-2467-4c0a-9d65-9dc650766e14","Type":"ContainerStarted","Data":"e9d385419ac89a720d8fe9b9abdf812a4cae4cda0c2add5384b1c1678e15cefd"} Jul 07 16:17:42.820465 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:42.820061 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:42.848417 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:42.848365 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" podStartSLOduration=1.474890485 podStartE2EDuration="1.848346522s" totalImagesPullingTime="373.456037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:17:42.847652162 +0000 UTC m=+1997.339370627" watchObservedRunningTime="2026-07-07 16:17:42.848346522 +0000 UTC m=+1997.340064985" Jul 07 16:17:48.831215 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:48.831175 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:17:54.850188 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:54.850134 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8"] Jul 07 16:17:54.850704 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:54.850526 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" podUID="e9b01360-28f2-45ce-a9e2-ab450ba87484" containerName="maas-api" containerID="cri-o://5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374" gracePeriod=30 Jul 07 16:17:55.104082 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.104002 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:55.201953 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.201920 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4qtn\" (UniqueName: \"kubernetes.io/projected/e9b01360-28f2-45ce-a9e2-ab450ba87484-kube-api-access-v4qtn\") pod \"e9b01360-28f2-45ce-a9e2-ab450ba87484\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " Jul 07 16:17:55.202137 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.202005 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9b01360-28f2-45ce-a9e2-ab450ba87484-openshift-service-ca\") pod \"e9b01360-28f2-45ce-a9e2-ab450ba87484\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " Jul 07 16:17:55.202137 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.202045 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls\") pod \"e9b01360-28f2-45ce-a9e2-ab450ba87484\" (UID: \"e9b01360-28f2-45ce-a9e2-ab450ba87484\") " Jul 07 16:17:55.202470 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.202428 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9b01360-28f2-45ce-a9e2-ab450ba87484-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9b01360-28f2-45ce-a9e2-ab450ba87484" (UID: "e9b01360-28f2-45ce-a9e2-ab450ba87484"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:17:55.204469 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.204443 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9b01360-28f2-45ce-a9e2-ab450ba87484-kube-api-access-v4qtn" (OuterVolumeSpecName: "kube-api-access-v4qtn") pod "e9b01360-28f2-45ce-a9e2-ab450ba87484" (UID: "e9b01360-28f2-45ce-a9e2-ab450ba87484"). InnerVolumeSpecName "kube-api-access-v4qtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:17:55.204600 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.204577 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9b01360-28f2-45ce-a9e2-ab450ba87484" (UID: "e9b01360-28f2-45ce-a9e2-ab450ba87484"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:17:55.303407 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.303359 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4qtn\" (UniqueName: \"kubernetes.io/projected/e9b01360-28f2-45ce-a9e2-ab450ba87484-kube-api-access-v4qtn\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:17:55.303407 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.303396 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9b01360-28f2-45ce-a9e2-ab450ba87484-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:17:55.303407 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.303408 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9b01360-28f2-45ce-a9e2-ab450ba87484-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:17:55.889316 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.889276 2585 generic.go:356] "Generic (PLEG): container finished" podID="e9b01360-28f2-45ce-a9e2-ab450ba87484" containerID="5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374" exitCode=0 Jul 07 16:17:55.889741 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.889333 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" Jul 07 16:17:55.889741 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.889369 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" event={"ID":"e9b01360-28f2-45ce-a9e2-ab450ba87484","Type":"ContainerDied","Data":"5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374"} Jul 07 16:17:55.889741 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.889433 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8" event={"ID":"e9b01360-28f2-45ce-a9e2-ab450ba87484","Type":"ContainerDied","Data":"0a4d248c8e1c51f85837ed32ac7386d97bc76363c1a83b39b1443d10fc6dc7b5"} Jul 07 16:17:55.889741 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.889455 2585 scope.go:117] "RemoveContainer" containerID="5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374" Jul 07 16:17:55.899343 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.899320 2585 scope.go:117] "RemoveContainer" containerID="5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374" Jul 07 16:17:55.899671 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:17:55.899649 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374\": container with ID starting with 5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374 not found: ID does not exist" containerID="5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374" Jul 07 16:17:55.899756 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.899683 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374"} err="failed to get container status \"5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374\": rpc error: code = NotFound desc = could not find container \"5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374\": container with ID starting with 5395a44db7925a96f4b57ccaaacc5511ef0175052d5c97c4358e05edbcab7374 not found: ID does not exist" Jul 07 16:17:55.912912 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.912873 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8"] Jul 07 16:17:55.917241 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:55.917210 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1dcebd-868d94999-hfgj8"] Jul 07 16:17:56.031030 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:17:56.030997 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9b01360-28f2-45ce-a9e2-ab450ba87484" path="/var/lib/kubelet/pods/e9b01360-28f2-45ce-a9e2-ab450ba87484/volumes" Jul 07 16:18:04.045847 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.045805 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d"] Jul 07 16:18:04.046418 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.046033 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" podUID="a07d1556-2467-4c0a-9d65-9dc650766e14" containerName="maas-api" containerID="cri-o://87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1" gracePeriod=30 Jul 07 16:18:04.314374 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.314343 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:18:04.388843 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.388804 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d1556-2467-4c0a-9d65-9dc650766e14-openshift-service-ca\") pod \"a07d1556-2467-4c0a-9d65-9dc650766e14\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " Jul 07 16:18:04.389026 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.388969 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d1556-2467-4c0a-9d65-9dc650766e14-maas-api-tls\") pod \"a07d1556-2467-4c0a-9d65-9dc650766e14\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " Jul 07 16:18:04.389026 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.389012 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxj5z\" (UniqueName: \"kubernetes.io/projected/a07d1556-2467-4c0a-9d65-9dc650766e14-kube-api-access-cxj5z\") pod \"a07d1556-2467-4c0a-9d65-9dc650766e14\" (UID: \"a07d1556-2467-4c0a-9d65-9dc650766e14\") " Jul 07 16:18:04.389299 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.389275 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07d1556-2467-4c0a-9d65-9dc650766e14-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a07d1556-2467-4c0a-9d65-9dc650766e14" (UID: "a07d1556-2467-4c0a-9d65-9dc650766e14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:18:04.391440 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.391399 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a07d1556-2467-4c0a-9d65-9dc650766e14-kube-api-access-cxj5z" (OuterVolumeSpecName: "kube-api-access-cxj5z") pod "a07d1556-2467-4c0a-9d65-9dc650766e14" (UID: "a07d1556-2467-4c0a-9d65-9dc650766e14"). InnerVolumeSpecName "kube-api-access-cxj5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:18:04.391588 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.391473 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a07d1556-2467-4c0a-9d65-9dc650766e14-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a07d1556-2467-4c0a-9d65-9dc650766e14" (UID: "a07d1556-2467-4c0a-9d65-9dc650766e14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:18:04.490209 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.490147 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d1556-2467-4c0a-9d65-9dc650766e14-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:18:04.490209 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.490204 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxj5z\" (UniqueName: \"kubernetes.io/projected/a07d1556-2467-4c0a-9d65-9dc650766e14-kube-api-access-cxj5z\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:18:04.490209 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.490219 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d1556-2467-4c0a-9d65-9dc650766e14-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:18:04.938280 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.938239 2585 generic.go:356] "Generic (PLEG): container finished" podID="a07d1556-2467-4c0a-9d65-9dc650766e14" containerID="87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1" exitCode=0 Jul 07 16:18:04.938503 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.938296 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" Jul 07 16:18:04.938503 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.938329 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" event={"ID":"a07d1556-2467-4c0a-9d65-9dc650766e14","Type":"ContainerDied","Data":"87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1"} Jul 07 16:18:04.938503 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.938420 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d" event={"ID":"a07d1556-2467-4c0a-9d65-9dc650766e14","Type":"ContainerDied","Data":"e9d385419ac89a720d8fe9b9abdf812a4cae4cda0c2add5384b1c1678e15cefd"} Jul 07 16:18:04.938503 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.938451 2585 scope.go:117] "RemoveContainer" containerID="87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1" Jul 07 16:18:04.948069 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.948048 2585 scope.go:117] "RemoveContainer" containerID="87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1" Jul 07 16:18:04.948402 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:18:04.948384 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1\": container with ID starting with 87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1 not found: ID does not exist" containerID="87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1" Jul 07 16:18:04.948476 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.948419 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1"} err="failed to get container status \"87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1\": rpc error: code = NotFound desc = could not find container \"87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1\": container with ID starting with 87570155330f29f04921445018326f75e30566381b701b54b468853c8718b9b1 not found: ID does not exist" Jul 07 16:18:04.987960 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.987925 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d"] Jul 07 16:18:04.992687 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:04.992655 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e60798-6884bb458b-mkx8d"] Jul 07 16:18:06.031805 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:06.031767 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a07d1556-2467-4c0a-9d65-9dc650766e14" path="/var/lib/kubelet/pods/a07d1556-2467-4c0a-9d65-9dc650766e14/volumes" Jul 07 16:18:13.658262 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658223 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5"] Jul 07 16:18:13.658752 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658667 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07d1556-2467-4c0a-9d65-9dc650766e14" containerName="maas-api" Jul 07 16:18:13.658752 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658683 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07d1556-2467-4c0a-9d65-9dc650766e14" containerName="maas-api" Jul 07 16:18:13.658752 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658715 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9b01360-28f2-45ce-a9e2-ab450ba87484" containerName="maas-api" Jul 07 16:18:13.658752 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658723 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9b01360-28f2-45ce-a9e2-ab450ba87484" containerName="maas-api" Jul 07 16:18:13.658967 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658805 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9b01360-28f2-45ce-a9e2-ab450ba87484" containerName="maas-api" Jul 07 16:18:13.658967 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.658817 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a07d1556-2467-4c0a-9d65-9dc650766e14" containerName="maas-api" Jul 07 16:18:13.661913 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.661894 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.664815 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.664785 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-94a996-serving-cert\"" Jul 07 16:18:13.676332 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.676299 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5"] Jul 07 16:18:13.786410 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.786372 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76f6r\" (UniqueName: \"kubernetes.io/projected/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-kube-api-access-76f6r\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.786621 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.786438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.786621 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.786536 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.887121 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.887073 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.887354 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.887141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.887354 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.887239 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76f6r\" (UniqueName: \"kubernetes.io/projected/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-kube-api-access-76f6r\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.887354 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:18:13.887307 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-serving-cert: secret "maas-api-e2e-shared-a-94a996-serving-cert" not found Jul 07 16:18:13.887498 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:18:13.887387 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls podName:4b9823cb-b2e0-4f8b-ad85-486b9784db3a nodeName:}" failed. No retries permitted until 2026-07-07 16:18:14.387365458 +0000 UTC m=+2028.879083916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls") pod "maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" (UID: "4b9823cb-b2e0-4f8b-ad85-486b9784db3a") : secret "maas-api-e2e-shared-a-94a996-serving-cert" not found Jul 07 16:18:13.887789 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.887771 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:13.906686 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:13.906655 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76f6r\" (UniqueName: \"kubernetes.io/projected/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-kube-api-access-76f6r\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:14.392390 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:14.392346 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:14.395218 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:14.395184 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-94a996-c667d6877-fqzz5\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:14.572755 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:14.572708 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:14.725003 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:14.724966 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5"] Jul 07 16:18:14.728310 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:18:14.728279 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9823cb_b2e0_4f8b_ad85_486b9784db3a.slice/crio-1c81cfd6cdb2373cdbc504ee3a179208fbcb814c5ad2a9f085ebdeacc605f239 WatchSource:0}: Error finding container 1c81cfd6cdb2373cdbc504ee3a179208fbcb814c5ad2a9f085ebdeacc605f239: Status 404 returned error can't find the container with id 1c81cfd6cdb2373cdbc504ee3a179208fbcb814c5ad2a9f085ebdeacc605f239 Jul 07 16:18:14.992947 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:14.992913 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" event={"ID":"4b9823cb-b2e0-4f8b-ad85-486b9784db3a","Type":"ContainerStarted","Data":"1c81cfd6cdb2373cdbc504ee3a179208fbcb814c5ad2a9f085ebdeacc605f239"} Jul 07 16:18:15.999815 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:15.999771 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" event={"ID":"4b9823cb-b2e0-4f8b-ad85-486b9784db3a","Type":"ContainerStarted","Data":"dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018"} Jul 07 16:18:16.000320 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:15.999835 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:16.020516 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:16.020457 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" podStartSLOduration=2.614972034 podStartE2EDuration="3.020440352s" totalImagesPullingTime="405.468318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:16.01814372 +0000 UTC m=+2030.509862183" watchObservedRunningTime="2026-07-07 16:18:16.020440352 +0000 UTC m=+2030.512158815" Jul 07 16:18:21.804379 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.804340 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf"] Jul 07 16:18:21.808330 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.808299 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:21.811403 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.811365 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-1cea82-serving-cert\"" Jul 07 16:18:21.823319 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.823288 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf"] Jul 07 16:18:21.973222 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.973171 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7062e561-8ec8-4781-a53f-61facbf1645d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:21.973406 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.973250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7062e561-8ec8-4781-a53f-61facbf1645d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:21.973406 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:21.973291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmpfn\" (UniqueName: \"kubernetes.io/projected/7062e561-8ec8-4781-a53f-61facbf1645d-kube-api-access-wmpfn\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.011670 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.011637 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:18:22.074585 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.074488 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7062e561-8ec8-4781-a53f-61facbf1645d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.074585 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.074543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7062e561-8ec8-4781-a53f-61facbf1645d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.074585 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.074577 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmpfn\" (UniqueName: \"kubernetes.io/projected/7062e561-8ec8-4781-a53f-61facbf1645d-kube-api-access-wmpfn\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.075153 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.075127 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7062e561-8ec8-4781-a53f-61facbf1645d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.077209 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.077148 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7062e561-8ec8-4781-a53f-61facbf1645d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.084739 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.084707 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmpfn\" (UniqueName: \"kubernetes.io/projected/7062e561-8ec8-4781-a53f-61facbf1645d-kube-api-access-wmpfn\") pod \"maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.119604 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.119570 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:22.273966 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:22.273934 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf"] Jul 07 16:18:22.276346 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:18:22.276314 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7062e561_8ec8_4781_a53f_61facbf1645d.slice/crio-60668b6d7f3195312dbfdfeb91c99effd0f9fc02a39e679682213575018d5dac WatchSource:0}: Error finding container 60668b6d7f3195312dbfdfeb91c99effd0f9fc02a39e679682213575018d5dac: Status 404 returned error can't find the container with id 60668b6d7f3195312dbfdfeb91c99effd0f9fc02a39e679682213575018d5dac Jul 07 16:18:23.039327 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:23.039290 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" event={"ID":"7062e561-8ec8-4781-a53f-61facbf1645d","Type":"ContainerStarted","Data":"4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4"} Jul 07 16:18:23.039767 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:23.039349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" event={"ID":"7062e561-8ec8-4781-a53f-61facbf1645d","Type":"ContainerStarted","Data":"60668b6d7f3195312dbfdfeb91c99effd0f9fc02a39e679682213575018d5dac"} Jul 07 16:18:23.039767 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:23.039394 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:18:23.070968 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:23.070911 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" podStartSLOduration=1.691874587 podStartE2EDuration="2.070893158s" totalImagesPullingTime="379.018571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:23.069345975 +0000 UTC m=+2037.561064437" watchObservedRunningTime="2026-07-07 16:18:23.070893158 +0000 UTC m=+2037.562611627" Jul 07 16:18:29.050314 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:18:29.050284 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:24:39.884413 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:39.884375 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5"] Jul 07 16:24:39.884887 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:39.884627 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" podUID="4b9823cb-b2e0-4f8b-ad85-486b9784db3a" containerName="maas-api" containerID="cri-o://dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018" gracePeriod=30 Jul 07 16:24:40.146394 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.146333 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:24:40.210947 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.210911 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls\") pod \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " Jul 07 16:24:40.211128 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.211043 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-openshift-service-ca\") pod \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " Jul 07 16:24:40.211128 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.211107 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76f6r\" (UniqueName: \"kubernetes.io/projected/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-kube-api-access-76f6r\") pod \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\" (UID: \"4b9823cb-b2e0-4f8b-ad85-486b9784db3a\") " Jul 07 16:24:40.211444 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.211415 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b9823cb-b2e0-4f8b-ad85-486b9784db3a" (UID: "4b9823cb-b2e0-4f8b-ad85-486b9784db3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:24:40.211598 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.211533 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:24:40.213364 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.213328 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-kube-api-access-76f6r" (OuterVolumeSpecName: "kube-api-access-76f6r") pod "4b9823cb-b2e0-4f8b-ad85-486b9784db3a" (UID: "4b9823cb-b2e0-4f8b-ad85-486b9784db3a"). InnerVolumeSpecName "kube-api-access-76f6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:24:40.213489 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.213373 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b9823cb-b2e0-4f8b-ad85-486b9784db3a" (UID: "4b9823cb-b2e0-4f8b-ad85-486b9784db3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:24:40.312268 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.312218 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:24:40.312268 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.312274 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76f6r\" (UniqueName: \"kubernetes.io/projected/4b9823cb-b2e0-4f8b-ad85-486b9784db3a-kube-api-access-76f6r\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:24:40.929795 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.929760 2585 generic.go:356] "Generic (PLEG): container finished" podID="4b9823cb-b2e0-4f8b-ad85-486b9784db3a" containerID="dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018" exitCode=0 Jul 07 16:24:40.930238 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.929807 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" Jul 07 16:24:40.930238 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.929848 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" event={"ID":"4b9823cb-b2e0-4f8b-ad85-486b9784db3a","Type":"ContainerDied","Data":"dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018"} Jul 07 16:24:40.930238 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.929907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5" event={"ID":"4b9823cb-b2e0-4f8b-ad85-486b9784db3a","Type":"ContainerDied","Data":"1c81cfd6cdb2373cdbc504ee3a179208fbcb814c5ad2a9f085ebdeacc605f239"} Jul 07 16:24:40.930238 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.929932 2585 scope.go:117] "RemoveContainer" containerID="dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018" Jul 07 16:24:40.941424 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.941406 2585 scope.go:117] "RemoveContainer" containerID="dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018" Jul 07 16:24:40.941698 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:24:40.941675 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018\": container with ID starting with dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018 not found: ID does not exist" containerID="dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018" Jul 07 16:24:40.941750 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.941708 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018"} err="failed to get container status \"dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018\": rpc error: code = NotFound desc = could not find container \"dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018\": container with ID starting with dd3434bf787c98c47d0818c309d604db273c4743d7b31b05c216c6b1bbb89018 not found: ID does not exist" Jul 07 16:24:40.957775 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.957752 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5"] Jul 07 16:24:40.962398 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:40.962376 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94a996-c667d6877-fqzz5"] Jul 07 16:24:42.031025 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:24:42.030988 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b9823cb-b2e0-4f8b-ad85-486b9784db3a" path="/var/lib/kubelet/pods/4b9823cb-b2e0-4f8b-ad85-486b9784db3a/volumes" Jul 07 16:25:21.074511 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.074477 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf"] Jul 07 16:25:21.075005 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.074690 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" podUID="7062e561-8ec8-4781-a53f-61facbf1645d" containerName="maas-api" containerID="cri-o://4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4" gracePeriod=30 Jul 07 16:25:21.325465 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.325407 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:25:21.429772 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.429741 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmpfn\" (UniqueName: \"kubernetes.io/projected/7062e561-8ec8-4781-a53f-61facbf1645d-kube-api-access-wmpfn\") pod \"7062e561-8ec8-4781-a53f-61facbf1645d\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " Jul 07 16:25:21.429958 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.429813 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7062e561-8ec8-4781-a53f-61facbf1645d-openshift-service-ca\") pod \"7062e561-8ec8-4781-a53f-61facbf1645d\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " Jul 07 16:25:21.429958 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.429842 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7062e561-8ec8-4781-a53f-61facbf1645d-maas-api-tls\") pod \"7062e561-8ec8-4781-a53f-61facbf1645d\" (UID: \"7062e561-8ec8-4781-a53f-61facbf1645d\") " Jul 07 16:25:21.430135 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.430111 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7062e561-8ec8-4781-a53f-61facbf1645d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7062e561-8ec8-4781-a53f-61facbf1645d" (UID: "7062e561-8ec8-4781-a53f-61facbf1645d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:25:21.430246 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.430228 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7062e561-8ec8-4781-a53f-61facbf1645d-openshift-service-ca\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:25:21.432130 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.432102 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7062e561-8ec8-4781-a53f-61facbf1645d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7062e561-8ec8-4781-a53f-61facbf1645d" (UID: "7062e561-8ec8-4781-a53f-61facbf1645d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:25:21.432245 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.432221 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7062e561-8ec8-4781-a53f-61facbf1645d-kube-api-access-wmpfn" (OuterVolumeSpecName: "kube-api-access-wmpfn") pod "7062e561-8ec8-4781-a53f-61facbf1645d" (UID: "7062e561-8ec8-4781-a53f-61facbf1645d"). InnerVolumeSpecName "kube-api-access-wmpfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:25:21.531471 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.531431 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7062e561-8ec8-4781-a53f-61facbf1645d-maas-api-tls\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:25:21.531471 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:21.531464 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmpfn\" (UniqueName: \"kubernetes.io/projected/7062e561-8ec8-4781-a53f-61facbf1645d-kube-api-access-wmpfn\") on node \"ip-10-0-136-171.ec2.internal\" DevicePath \"\"" Jul 07 16:25:22.136494 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.136455 2585 generic.go:356] "Generic (PLEG): container finished" podID="7062e561-8ec8-4781-a53f-61facbf1645d" containerID="4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4" exitCode=0 Jul 07 16:25:22.136913 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.136504 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" Jul 07 16:25:22.136913 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.136542 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" event={"ID":"7062e561-8ec8-4781-a53f-61facbf1645d","Type":"ContainerDied","Data":"4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4"} Jul 07 16:25:22.136913 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.136593 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf" event={"ID":"7062e561-8ec8-4781-a53f-61facbf1645d","Type":"ContainerDied","Data":"60668b6d7f3195312dbfdfeb91c99effd0f9fc02a39e679682213575018d5dac"} Jul 07 16:25:22.136913 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.136621 2585 scope.go:117] "RemoveContainer" containerID="4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4" Jul 07 16:25:22.145417 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.145399 2585 scope.go:117] "RemoveContainer" containerID="4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4" Jul 07 16:25:22.145659 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:25:22.145641 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4\": container with ID starting with 4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4 not found: ID does not exist" containerID="4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4" Jul 07 16:25:22.145705 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.145668 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4"} err="failed to get container status \"4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4\": rpc error: code = NotFound desc = could not find container \"4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4\": container with ID starting with 4183fe27cee7f46dcb2243f08f22263c70e49858bcc4a082843f034c8c6971c4 not found: ID does not exist" Jul 07 16:25:22.155661 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.155636 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf"] Jul 07 16:25:22.163370 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:22.163349 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1cea82-69b8f68bb4-h9slf"] Jul 07 16:25:24.030611 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:24.030576 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7062e561-8ec8-4781-a53f-61facbf1645d" path="/var/lib/kubelet/pods/7062e561-8ec8-4781-a53f-61facbf1645d/volumes" Jul 07 16:25:32.966355 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:32.966319 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-796b9b6788-85hkb_0d13d56b-4b90-4f93-8331-9da4702c85b2/manager/0.log" Jul 07 16:25:33.315445 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:33.315419 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-2cszn_6adaf563-7f1f-44ff-835b-025ce7cd5a53/copy-manifests/0.log" Jul 07 16:25:33.327141 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:33.327114 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-2cszn_6adaf563-7f1f-44ff-835b-025ce7cd5a53/manager/0.log" Jul 07 16:25:35.224114 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:35.224087 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-k5nhn_e1ecb404-786b-4e09-afe1-375c8b87cc44/kuadrant-console-plugin/0.log" Jul 07 16:25:36.269503 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:36.269475 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59ffcfbcb9-l4glh_3c010cab-e547-4c0d-ab2c-bf36468f51f6/kube-auth-proxy/0.log" Jul 07 16:25:37.229856 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:37.229826 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-chnr2_3ae2f79e-9951-4ee4-b98a-e93a0400014a/main/0.log" Jul 07 16:25:37.340033 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:37.340006 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-tgwvs_a2842970-2e1f-4b20-b285-281e71feb6e6/main/0.log" Jul 07 16:25:37.570944 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:37.570913 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmz2l2_4c08b0e5-2635-4ea7-bab6-b9c7031c075e/main/0.log" Jul 07 16:25:45.409608 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:45.409576 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dl6lx_234a4877-fd36-4a8e-923e-24b3bee4b1bb/global-pull-secret-syncer/0.log" Jul 07 16:25:45.605883 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:45.605846 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wn9wc_8ea2a280-4017-40a2-ae11-946fe06fe636/konnectivity-agent/0.log" Jul 07 16:25:45.632257 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:45.632218 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-171.ec2.internal_72397af2de8d127b4272a28e33dc0313/haproxy/0.log" Jul 07 16:25:51.357128 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:51.357104 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-k5nhn_e1ecb404-786b-4e09-afe1-375c8b87cc44/kuadrant-console-plugin/0.log" Jul 07 16:25:53.388607 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.388527 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/alertmanager/0.log" Jul 07 16:25:53.426831 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.426808 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/config-reloader/0.log" Jul 07 16:25:53.458241 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.458217 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/kube-rbac-proxy-web/0.log" Jul 07 16:25:53.487131 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.487106 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/kube-rbac-proxy/0.log" Jul 07 16:25:53.516788 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.516766 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/kube-rbac-proxy-metric/0.log" Jul 07 16:25:53.547171 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.547147 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/prom-label-proxy/0.log" Jul 07 16:25:53.576076 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.576053 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_143bf90f-71c5-4af3-aeda-f7440be81da0/init-config-reloader/0.log" Jul 07 16:25:53.653445 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.653361 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-nspm8_59b448ea-38f8-4aec-aa8a-d379b07accff/kube-state-metrics/0.log" Jul 07 16:25:53.681250 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.681229 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-nspm8_59b448ea-38f8-4aec-aa8a-d379b07accff/kube-rbac-proxy-main/0.log" Jul 07 16:25:53.708063 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.708037 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-nspm8_59b448ea-38f8-4aec-aa8a-d379b07accff/kube-rbac-proxy-self/0.log" Jul 07 16:25:53.738670 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.738645 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-549f79f6ff-bvd77_bb9fd8da-8f8c-47ec-bd66-f454929fa434/metrics-server/0.log" Jul 07 16:25:53.766989 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.766968 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-mq9tb_4933bd4b-56b2-4a4a-867a-4141f71cc75d/monitoring-plugin/0.log" Jul 07 16:25:53.817913 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.817880 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5bzj_6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3/node-exporter/0.log" Jul 07 16:25:53.843185 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.843123 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5bzj_6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3/kube-rbac-proxy/0.log" Jul 07 16:25:53.872301 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:53.872271 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5bzj_6f2e3e28-4afc-41b5-a6d0-32fd6558c8a3/init-textfile/0.log" Jul 07 16:25:54.080683 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.080650 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nz7cp_29156cfa-8705-442d-9ee7-32719a333b8c/kube-rbac-proxy-main/0.log" Jul 07 16:25:54.106967 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.106929 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nz7cp_29156cfa-8705-442d-9ee7-32719a333b8c/kube-rbac-proxy-self/0.log" Jul 07 16:25:54.136552 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.136526 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nz7cp_29156cfa-8705-442d-9ee7-32719a333b8c/openshift-state-metrics/0.log" Jul 07 16:25:54.177407 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.177375 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/prometheus/0.log" Jul 07 16:25:54.202858 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.202818 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/config-reloader/0.log" Jul 07 16:25:54.238403 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.238379 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/thanos-sidecar/0.log" Jul 07 16:25:54.266604 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.266576 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/kube-rbac-proxy-web/0.log" Jul 07 16:25:54.293591 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.293568 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/kube-rbac-proxy/0.log" Jul 07 16:25:54.356886 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.356789 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/kube-rbac-proxy-thanos/0.log" Jul 07 16:25:54.423178 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.423128 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881fe027-2e23-4bfe-a2c6-0ea1281d8610/init-config-reloader/0.log" Jul 07 16:25:54.498617 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.498593 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-fcqxv_cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06/prometheus-operator/0.log" Jul 07 16:25:54.523659 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.523631 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-fcqxv_cb9c9816-4c5b-49fa-a4ec-ec7fa1826c06/kube-rbac-proxy/0.log" Jul 07 16:25:54.591741 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.591714 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5ddd88968-p27t2_b51a8036-5a08-4fe1-9683-793eb3e26376/telemeter-client/0.log" Jul 07 16:25:54.624536 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.624460 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5ddd88968-p27t2_b51a8036-5a08-4fe1-9683-793eb3e26376/reload/0.log" Jul 07 16:25:54.650837 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.650811 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5ddd88968-p27t2_b51a8036-5a08-4fe1-9683-793eb3e26376/kube-rbac-proxy/0.log" Jul 07 16:25:54.691301 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.691274 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d68f8b574-62sm9_4e6258fa-5b77-4809-a472-fe0f23960f80/thanos-query/0.log" Jul 07 16:25:54.718388 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.718353 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d68f8b574-62sm9_4e6258fa-5b77-4809-a472-fe0f23960f80/kube-rbac-proxy-web/0.log" Jul 07 16:25:54.751132 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.751102 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d68f8b574-62sm9_4e6258fa-5b77-4809-a472-fe0f23960f80/kube-rbac-proxy/0.log" Jul 07 16:25:54.777452 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.777420 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d68f8b574-62sm9_4e6258fa-5b77-4809-a472-fe0f23960f80/prom-label-proxy/0.log" Jul 07 16:25:54.803208 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.803179 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d68f8b574-62sm9_4e6258fa-5b77-4809-a472-fe0f23960f80/kube-rbac-proxy-rules/0.log" Jul 07 16:25:54.830516 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:54.830485 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d68f8b574-62sm9_4e6258fa-5b77-4809-a472-fe0f23960f80/kube-rbac-proxy-metrics/0.log" Jul 07 16:25:55.379061 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379029 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g"] Jul 07 16:25:55.379499 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379478 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b9823cb-b2e0-4f8b-ad85-486b9784db3a" containerName="maas-api" Jul 07 16:25:55.379555 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379500 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b9823cb-b2e0-4f8b-ad85-486b9784db3a" containerName="maas-api" Jul 07 16:25:55.379555 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379520 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7062e561-8ec8-4781-a53f-61facbf1645d" containerName="maas-api" Jul 07 16:25:55.379555 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379525 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7062e561-8ec8-4781-a53f-61facbf1645d" containerName="maas-api" Jul 07 16:25:55.379665 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379584 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7062e561-8ec8-4781-a53f-61facbf1645d" containerName="maas-api" Jul 07 16:25:55.379665 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.379596 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b9823cb-b2e0-4f8b-ad85-486b9784db3a" containerName="maas-api" Jul 07 16:25:55.382806 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.382789 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.386311 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:25:55.386283 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-s9dw8\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-s9dw8\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 07 16:25:55.386311 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:25:55.386285 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-s9dw8\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-s9dw8\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 07 16:25:55.386573 ip-10-0-136-171 kubenswrapper[2585]: E0707 16:25:55.386306 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-9fs8h\" is forbidden: User \"system:node:ip-10-0-136-171.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-must-gather-s9dw8\": no relationship found between node 'ip-10-0-136-171.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-s9dw8\"/\"default-dockercfg-9fs8h\"" type="*v1.Secret" Jul 07 16:25:55.397237 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.397205 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g"] Jul 07 16:25:55.476621 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.476581 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-proc\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.477019 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.476665 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-podres\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.477019 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.476697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-sys\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.477019 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.476733 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxfqp\" (UniqueName: \"kubernetes.io/projected/1778cd11-9479-4b85-a73c-7ca7760f27ae-kube-api-access-nxfqp\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.477019 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.476783 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-lib-modules\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577507 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577453 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxfqp\" (UniqueName: \"kubernetes.io/projected/1778cd11-9479-4b85-a73c-7ca7760f27ae-kube-api-access-nxfqp\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577689 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577522 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-lib-modules\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577689 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577569 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-proc\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577689 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577620 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-podres\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577689 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577646 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-sys\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577866 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577682 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-proc\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577866 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577733 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-sys\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577866 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577730 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-lib-modules\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:55.577866 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:55.577801 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1778cd11-9479-4b85-a73c-7ca7760f27ae-podres\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:56.378448 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:56.378412 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9dw8\"/\"kube-root-ca.crt\"" Jul 07 16:25:56.589100 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:56.589066 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s9dw8\"/\"default-dockercfg-9fs8h\"" Jul 07 16:25:56.938562 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:56.938533 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9dw8\"/\"openshift-service-ca.crt\"" Jul 07 16:25:56.948040 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:56.948001 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxfqp\" (UniqueName: \"kubernetes.io/projected/1778cd11-9479-4b85-a73c-7ca7760f27ae-kube-api-access-nxfqp\") pod \"perf-node-gather-daemonset-ztd9g\" (UID: \"1778cd11-9479-4b85-a73c-7ca7760f27ae\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:57.193287 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:57.193182 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:57.338383 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:57.338356 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g"] Jul 07 16:25:57.340217 ip-10-0-136-171 kubenswrapper[2585]: W0707 16:25:57.340187 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1778cd11_9479_4b85_a73c_7ca7760f27ae.slice/crio-0f48f35ce02c5bf2f5e6380a92d758fe3b3a7b63c965ae9299e0ba8e18354602 WatchSource:0}: Error finding container 0f48f35ce02c5bf2f5e6380a92d758fe3b3a7b63c965ae9299e0ba8e18354602: Status 404 returned error can't find the container with id 0f48f35ce02c5bf2f5e6380a92d758fe3b3a7b63c965ae9299e0ba8e18354602 Jul 07 16:25:58.316017 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:58.315978 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" event={"ID":"1778cd11-9479-4b85-a73c-7ca7760f27ae","Type":"ContainerStarted","Data":"487f742b4116366bbb5686929020c38e573f93af8e31f7020e0d2ab0ef21eb07"} Jul 07 16:25:58.316556 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:58.316047 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" event={"ID":"1778cd11-9479-4b85-a73c-7ca7760f27ae","Type":"ContainerStarted","Data":"0f48f35ce02c5bf2f5e6380a92d758fe3b3a7b63c965ae9299e0ba8e18354602"} Jul 07 16:25:58.316556 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:58.316077 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:25:58.336148 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:58.336092 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" podStartSLOduration=3.336076398 podStartE2EDuration="3.336076398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:58.334349574 +0000 UTC m=+2492.826068050" watchObservedRunningTime="2026-07-07 16:25:58.336076398 +0000 UTC m=+2492.827794917" Jul 07 16:25:59.119651 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:59.119624 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vnw9p_983ffad6-c30c-4d1b-9d7c-2c9c611f7f97/dns/0.log" Jul 07 16:25:59.151068 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:59.151040 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vnw9p_983ffad6-c30c-4d1b-9d7c-2c9c611f7f97/kube-rbac-proxy/0.log" Jul 07 16:25:59.325055 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:59.325027 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z82v7_27198ddb-32a7-430e-a846-a61e18ba37e3/dns-node-resolver/0.log" Jul 07 16:25:59.947682 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:25:59.947635 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dxr9g_88836d19-83c3-4e9f-bb58-e223acbc47d4/node-ca/0.log" Jul 07 16:26:01.246753 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:01.246725 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59ffcfbcb9-l4glh_3c010cab-e547-4c0d-ab2c-bf36468f51f6/kube-auth-proxy/0.log" Jul 07 16:26:02.279763 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:02.279723 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k6t8d_8e224e69-6838-49f7-b568-4cdb20c1a653/serve-healthcheck-canary/0.log" Jul 07 16:26:03.069408 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:03.069377 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zscjv_57b264b8-d138-46d3-ae42-45fb48a33dba/kube-rbac-proxy/0.log" Jul 07 16:26:03.094794 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:03.094765 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zscjv_57b264b8-d138-46d3-ae42-45fb48a33dba/exporter/0.log" Jul 07 16:26:03.121604 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:03.121576 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zscjv_57b264b8-d138-46d3-ae42-45fb48a33dba/extractor/0.log" Jul 07 16:26:04.331335 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:04.331306 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-ztd9g" Jul 07 16:26:05.924831 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:05.924795 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-796b9b6788-85hkb_0d13d56b-4b90-4f93-8331-9da4702c85b2/manager/0.log" Jul 07 16:26:06.077751 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:06.077721 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-2cszn_6adaf563-7f1f-44ff-835b-025ce7cd5a53/manager/0.log" Jul 07 16:26:06.111868 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:06.111840 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-2cszn_6adaf563-7f1f-44ff-835b-025ce7cd5a53/copy-manifests/0.log" Jul 07 16:26:06.590651 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:06.590597 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59ffcfbcb9-l4glh_3c010cab-e547-4c0d-ab2c-bf36468f51f6/kube-auth-proxy/0.log" Jul 07 16:26:07.679066 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:07.679040 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-646f7cc9db-x4bzv_ec0a02b6-47ee-46ae-9e2b-29c609c638a7/manager/0.log" Jul 07 16:26:07.711678 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:07.711648 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jwf6p_83cf9eb3-5f10-4be4-884f-e46404696d88/openshift-lws-operator/0.log" Jul 07 16:26:08.157378 ip-10-0-136-171 kubenswrapper[2585]: I0707 16:26:08.157345 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59ffcfbcb9-l4glh_3c010cab-e547-4c0d-ab2c-bf36468f51f6/kube-auth-proxy/0.log"