Jul 21 14:15:34.778665 ip-10-0-129-87 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 14:15:34.778676 ip-10-0-129-87 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 14:15:34.778683 ip-10-0-129-87 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 14:15:34.778952 ip-10-0-129-87 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 14:15:46.180342 ip-10-0-129-87 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 14:15:46.180359 ip-10-0-129-87 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 41f18f9be4b543f99d5f05d81c40f998 --
Jul 21 14:17:53.731702 ip-10-0-129-87 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 14:17:54.198684 ip-10-0-129-87 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 14:17:54.198684 ip-10-0-129-87 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 14:17:54.198684 ip-10-0-129-87 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 14:17:54.198684 ip-10-0-129-87 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 14:17:54.198684 ip-10-0-129-87 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 14:17:54.200092 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.199945    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208743    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208761    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208765    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208769    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208772    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208775    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:17:54.208769 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208778    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208781    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208784    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208787    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208790    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208792    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208795    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208798    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208802    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208804    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208808    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208812    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208816    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208818    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208821    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208824    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208827    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208829    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208832    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:17:54.209013 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208834    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208837    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208839    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208842    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208844    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208847    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208849    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208852    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208854    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208857    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208859    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208863    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208867    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208869    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208872    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208875    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208878    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208880    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208883    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:17:54.209493 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208885    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208888    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208890    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208893    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208895    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208898    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208900    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208903    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208905    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208908    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208910    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208913    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208916    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208919    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208921    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208923    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208926    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208928    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208931    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208933    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:17:54.209990 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208935    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208938    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208940    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208943    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208945    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208948    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208951    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208956    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208959    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208962    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208964    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208967    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208969    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208972    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208974    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208977    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208979    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208982    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208984    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208987    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:17:54.210491 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208989    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208992    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208994    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208996    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.208999    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209002    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209004    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209006    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209009    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209011    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209014    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209017    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209019    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209022    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:17:54.210960 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.209024    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211188    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211197    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211200    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211203    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211206    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211208    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211211    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211221    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211226    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211229    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211231    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211235    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211239    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211242    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211245    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211248    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211250    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211254    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:17:54.211305 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211256    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211259    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211261    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211264    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211267    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211270    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211272    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211275    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211277    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211279    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211282    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211286    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211288    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211291    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211294    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211299    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211302    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211305    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211308    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:17:54.211786 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211311    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211313    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211316    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211319    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211321    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211323    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211326    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211328    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211330    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211333    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211335    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211338    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211340    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211343    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211346    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211348    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211351    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211353    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211355    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211358    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:17:54.212266 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211360    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211363    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211365    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211367    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211370    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211372    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211374    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211377    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211379    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211382    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211384    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211387    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211391    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211393    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211395    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211398    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211401    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211403    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211405    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211408    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:17:54.212796 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211410    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211412    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211427    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211430    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211432    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211435    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211444    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211448    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211451    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211453    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211456    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211459    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211461    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211463    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211466    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211468    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211471    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211473    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211476    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211478    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:17:54.213285 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211481    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211483    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211877    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211882    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211884    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211887    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211889    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211892    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211894    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211897    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211899    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211902    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211904    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211907    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211909    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211912    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211914    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211916    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211922    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211925    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:17:54.213790 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211928    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211932    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211936    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211938    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211941    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211943    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211946    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211948    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211950    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211953    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211955    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211958    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211961    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211963    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211966    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211968    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211970    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211973    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211975    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211977    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:17:54.214269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211980    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211982    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211984    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211987    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211989    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211991    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211994    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.211996    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212001    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212005    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212007    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212010    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212012    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212015    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212017    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212020    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212023    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212025    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212027    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:17:54.214777 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212030    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212032    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212035    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212037    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212039    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212042    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212045    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212048    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212050    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212052    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212055    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212058    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212060    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212062    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212064    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212067    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212069    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212072    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212074    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212076    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:17:54.215247 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212078    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212082    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212085    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212087    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212090    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212098    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212101    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212105    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212107    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212109    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212112    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212114    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212118    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212121    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212125    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212128    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212131    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212134    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212137    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:17:54.215727 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212140    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212143    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212145    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212215    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212222    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212231    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212235    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212239    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212242    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212247    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212250    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212255    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212258    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212262    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212267    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212270    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212273    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212276    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212279    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212283    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212286    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212289    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212292    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212295    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 14:17:54.216183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212298    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212301    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212304    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212307    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212309    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212312    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212315    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212318    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212321    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212324    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212327    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212331    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212333    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212336    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212339    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212341    2569 flags.go:64] FLAG: --enable-server="true"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212344    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212348    2569 flags.go:64] FLAG: --event-burst="100"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212351    2569 flags.go:64] FLAG: --event-qps="50"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212354    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212357    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212360    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212365    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212371    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212374    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 14:17:54.216758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212377    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212379    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212382    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212385    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212388    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212391    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212394    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212397    2569 flags.go:64] FLAG: --feature-gates=""
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212400    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212403    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212406    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212409    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212412    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212428    2569 flags.go:64] FLAG: --help="false"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212431    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212434    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212437    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212440    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212443    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212446    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212449    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212452    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212455    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 14:17:54.217365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212457    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212460    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212463    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212466    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212468    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212471    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212476    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212479    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212482    2569 flags.go:64] FLAG: --lock-file=""
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212485    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212488    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212491    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212496    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212499    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212502    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212505    2569 flags.go:64] FLAG: --logging-format="text"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212507    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212510    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212513    2569 flags.go:64] FLAG: --manifest-url=""
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212516    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212520    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212523    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212527    2569 flags.go:64] FLAG: --max-pods="110"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212530    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212532    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 14:17:54.218007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212535    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212538    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212541    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212543    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212546    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212553    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212556    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212558    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212562    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212564    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212569    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212572    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212575    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212579    2569 flags.go:64] FLAG: --port="10250"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212582    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212586    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07fd32db854bf8229"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212589    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212592    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212595    2569 flags.go:64] FLAG: --register-node="true"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212597    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212601    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212604    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212607    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212609    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212613    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 14:17:54.218689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212616    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212619    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212621    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212624    2569 flags.go:64] FLAG: --runonce="false"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212627    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212630    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212632    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212635    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212638    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212640    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212644    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212646    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212649    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212652    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212655    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212657    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212660    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212663    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212665    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212670    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212674    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212677    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212682    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212684    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212687    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 14:17:54.219306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212690    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212693    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212696    2569 flags.go:64] FLAG: --v="2"
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212700    2569 flags.go:64] FLAG: --version="false"
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212704    2569 flags.go:64] FLAG: --vmodule=""
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212708    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.212711    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212831    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212836    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212838    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212841    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212843    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212846    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212848    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212851    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212854    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212856    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212859    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212862    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212864    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212867    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212869    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:17:54.220054 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212871    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212874    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212877    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212879    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212882    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212886    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212888    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212892    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212894    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212896    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212899    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212902    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212904    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212906    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212909    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212912    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212914    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212917    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212919    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212922    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:17:54.220604 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212924    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212928    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212931    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212934    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212937    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212940    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212943    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212945    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212948    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212950    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212952    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212955    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212957    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212960    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212962    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212965    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212967    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212971    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212973    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:17:54.221105 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212977    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212980    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212982    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212985    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212987    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212990    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212992    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212995    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.212997    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213000    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213004    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213007    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213010    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213012    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213015    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213017    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213020    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213022    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213025    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:17:54.221706 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213027    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213030    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213032    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213035    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213037    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213040    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213042    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213045    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213047    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213050    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213052    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213056    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213058    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213061    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213064    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213068    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213072    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213074    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213077    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:17:54.222167 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213080    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213082    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213085    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213087    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213090    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213092    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.213095    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.214204    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.221586    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.221606    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.221626    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.221635    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221696    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221701    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221705    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:17:54.222700 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221708    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221711    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221714    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221717    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221719    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221722    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221725    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221727    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221731    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221735    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221738    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221741    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221744    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221747    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221749    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221751    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221754    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221756    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221759    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221761    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:17:54.223081 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221763    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221768    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221771    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221774    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221776    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221779    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221789    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221792    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221795    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221797    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221800    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221802    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221805    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221808    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221811    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221813    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221816    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221819    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221822    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221824    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:17:54.223598 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221826    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221829    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221831    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221834    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221838    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221841    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221843    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221846    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221849    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221851    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221853    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221856    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221859    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221861    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221863    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221866    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221868    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221870    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221874    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:17:54.224078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221882    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221884    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221887    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221889    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221892    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221894    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221897    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221899    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221902    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221904    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221907    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221909    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221911    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221914    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221916    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221919    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221935    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221938    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221941    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221944    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:17:54.224554 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221947    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221949    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221952    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221954    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221957    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221959    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221962    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221964    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221967    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221969    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221971    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221974    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221976    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221985    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221987    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221990    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:17:54.225062 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.221992    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.221997    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222151    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222156    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222159    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222161    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222164    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222167    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222170    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222173    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222175    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222178    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222181    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222184    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222187    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:17:54.225457 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222190    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222192    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222195    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222197    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222200    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222203    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222205    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222208    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222210    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222212    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222215    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222217    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222219    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222222    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222224    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222232    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222235    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222238    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222240    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222242    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:17:54.225875 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222245    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222247    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222250    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222252    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222254    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222257    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222261    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222264    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222267    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222269    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222272    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222274    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222277    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222279    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222281    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222284    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222286    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222288    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222291    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:17:54.226371 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222293    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222295    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222298    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222300    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222304    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222307    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222310    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222313    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222315    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222324    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222327    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222329    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222331    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222335    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222339    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222341    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222344    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222347    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222349    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:17:54.226901 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222352    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222354    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222357    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222359    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222362    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222364    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222366    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222369    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222371    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222374    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222376    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222378    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222380    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222383    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222385    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222388    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222390    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222393    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222396    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222398    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:17:54.227472 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222400    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222403    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222405    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222413    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222429    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222432    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222435    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:54.222437    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.222442    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.222559    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.225661    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.227168    2569 server.go:1019] "Starting client certificate rotation"
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.227293    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 14:17:54.227956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.227335    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 14:17:54.229955 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.229941    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 14:17:54.235096 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.235077    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 14:17:54.247285 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.247265    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 21 14:17:54.249364 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.249349    2569 log.go:25] "Validated CRI v1 image API"
Jul 21 14:17:54.250086 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.250073    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 14:17:54.253734 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.253705    2569 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ac707b64-8557-4008-aba6-e6462d7a973b:/dev/nvme0n1p3 c2224ad6-9a7c-4d96-8808-d112464e0c05:/dev/nvme0n1p4]
Jul 21 14:17:54.253793 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.253735    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 14:17:54.257846 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.257827    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 14:17:54.259641 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.259532    2569 manager.go:216] Machine: {Timestamp:2026-07-21 14:17:54.25787155 +0000 UTC m=+0.392266915 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099084 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a2c8b8eee88ee2f0aef75ffc83ea5 SystemUUID:ec2a2c8b-8eee-88ee-2f0a-ef75ffc83ea5 BootID:41f18f9b-e4b5-43f9-9d5f-05d81c40f998 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:47:86:06:33:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:47:86:06:33:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:49:b6:20:c4:9c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 14:17:54.259641 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.259639    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 14:17:54.259750 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.259718    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 14:17:54.263410 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.263383    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 14:17:54.263560 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.263413    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-87.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 14:17:54.264305 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.264295    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 14:17:54.264336 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.264308    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 14:17:54.264336 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.264316    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 14:17:54.265193 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.265182    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 14:17:54.266663 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.266653    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 14:17:54.269917 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.269905    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 14:17:54.270045 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270035    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 14:17:54.270074 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270068    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 14:17:54.270105 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270081    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 14:17:54.270105 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270093    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 14:17:54.270155 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270109    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 21 14:17:54.270155 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270119    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 14:17:54.270587 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270569    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 14:17:54.270654 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.270597    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 14:17:54.274030 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.274012    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 14:17:54.277609 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.277583    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 14:17:54.277711 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.277633    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 14:17:54.279881 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279867    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 14:17:54.279881 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279884    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279890    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279895    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279904    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279909    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279916    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279922    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279928    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279934    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279942    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 14:17:54.279982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.279947    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 14:17:54.280890 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.280880    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 14:17:54.280890 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.280889    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 14:17:54.281313 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.281291    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 14:17:54.281374 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.281337    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 14:17:54.283948 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.283933    2569 server.go:1268] "Started kubelet"
Jul 21 14:17:54.284067 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.284019    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 14:17:54.284120 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.284092    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 14:17:54.284252 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.284220    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 14:17:54.284748 ip-10-0-129-87 systemd[1]: Started Kubernetes Kubelet.
Jul 21 14:17:54.284862 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.284751    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vs8v6"
Jul 21 14:17:54.286096 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.286067    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 14:17:54.286449 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.286436    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 14:17:54.289082 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.289062    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-87.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 14:17:54.290022 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.288901    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-87.ec2.internal.18c453c8cf69429b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-87.ec2.internal,UID:ip-10-0-129-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-87.ec2.internal,},FirstTimestamp:2026-07-21 14:17:54.283905691 +0000 UTC m=+0.418301056,LastTimestamp:2026-07-21 14:17:54.283905691 +0000 UTC m=+0.418301056,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-87.ec2.internal,}"
Jul 21 14:17:54.290821 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.290792    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 14:17:54.293315 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.293296    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 14:17:54.293315 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.293302    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 14:17:54.293467 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.293453    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.294043 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294015    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 14:17:54.294043 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294016    2569 factory.go:55] Registering systemd factory
Jul 21 14:17:54.294202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294050    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 21 14:17:54.294202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294017    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 14:17:54.294202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294084    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 14:17:54.294202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294188    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 14:17:54.294202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294199    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 14:17:54.294445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294221    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vs8v6"
Jul 21 14:17:54.294445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294299    2569 factory.go:153] Registering CRI-O factory
Jul 21 14:17:54.294445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294318    2569 factory.go:223] Registration of the crio container factory successfully
Jul 21 14:17:54.294445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294368    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 14:17:54.294445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294391    2569 factory.go:103] Registering Raw factory
Jul 21 14:17:54.294445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294406    2569 manager.go:1196] Started watching for new ooms in manager
Jul 21 14:17:54.294999 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.294985    2569 manager.go:349] Starting recovery of all containers
Jul 21 14:17:54.299003 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.298979    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 14:17:54.299076 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.299021    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 14:17:54.305277 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.305176    2569 manager.go:354] Recovery completed
Jul 21 14:17:54.307688 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.307535    2569 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 21 14:17:54.310870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.310858    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.312615 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.312601    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.312678 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.312630    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.312678 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.312644    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.313073 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313060    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 14:17:54.313073 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313072    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 14:17:54.313148 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313087    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 14:17:54.313148 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313110    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.313826 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313811    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.313871 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313841    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.313871 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.313852    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.314230 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.314219    2569 policy_none.go:49] "None policy: Start"
Jul 21 14:17:54.314261 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.314235    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 14:17:54.314261 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.314245    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 14:17:54.315327 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.315317    2569 policy_none.go:47] "Start"
Jul 21 14:17:54.356781 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.356767    2569 manager.go:337] "Starting Device Plugin manager"
Jul 21 14:17:54.356852 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.356836    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 14:17:54.356852 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.356851    2569 server.go:87] "Starting device plugin registration server"
Jul 21 14:17:54.357118 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.357105    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 14:17:54.357207 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.357118    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 14:17:54.357257 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.357203    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 14:17:54.357306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.357275    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 14:17:54.357306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.357283    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 14:17:54.357852 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.357834    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 14:17:54.357943 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.357883    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.457939 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.457870    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.459052 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.458928    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.459052 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.458967    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.459052 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.458983    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.459052 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.459022    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.460530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.460496    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 14:17:54.461889 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.461873    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 14:17:54.461941 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.461917    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 14:17:54.461941 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.461941    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 14:17:54.462011 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.461982    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 14:17:54.465897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.465869    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:17:54.471805 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.471791    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.471857 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.471812    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-87.ec2.internal\": node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.519786 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.519760    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.562584 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.562554    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"]
Jul 21 14:17:54.562673 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.562631    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.563617 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.563598    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.563721 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.563635    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.563721 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.563648    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.565018 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565003    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.565148 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565134    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.565194 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565165    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.565783 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565761    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.565883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565791    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.565883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565799    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.565883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565807    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.565883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565816    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.565883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.565829    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.567198 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.567179    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.567299 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.567205    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:17:54.567852 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.567837    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:17:54.567919 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.567863    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:17:54.567919 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.567873    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:17:54.595666 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.595646    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/775ad56364ad1161b542cba6cf3f9498-config\") pod \"kube-apiserver-proxy-ip-10-0-129-87.ec2.internal\" (UID: \"775ad56364ad1161b542cba6cf3f9498\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.595765 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.595671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d975ce7bcfb76736e653a70dee3c187b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal\" (UID: \"d975ce7bcfb76736e653a70dee3c187b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.595765 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.595687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d975ce7bcfb76736e653a70dee3c187b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal\" (UID: \"d975ce7bcfb76736e653a70dee3c187b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.602233 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.602205    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-87.ec2.internal\" not found" node="ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.606659 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.606643    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-87.ec2.internal\" not found" node="ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.620228 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.620211    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.696138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.696086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/775ad56364ad1161b542cba6cf3f9498-config\") pod \"kube-apiserver-proxy-ip-10-0-129-87.ec2.internal\" (UID: \"775ad56364ad1161b542cba6cf3f9498\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.696138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.696134    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d975ce7bcfb76736e653a70dee3c187b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal\" (UID: \"d975ce7bcfb76736e653a70dee3c187b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.696334 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.696152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d975ce7bcfb76736e653a70dee3c187b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal\" (UID: \"d975ce7bcfb76736e653a70dee3c187b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.696334 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.696177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d975ce7bcfb76736e653a70dee3c187b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal\" (UID: \"d975ce7bcfb76736e653a70dee3c187b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.696334 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.696188    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/775ad56364ad1161b542cba6cf3f9498-config\") pod \"kube-apiserver-proxy-ip-10-0-129-87.ec2.internal\" (UID: \"775ad56364ad1161b542cba6cf3f9498\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.696334 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.696201    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d975ce7bcfb76736e653a70dee3c187b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal\" (UID: \"d975ce7bcfb76736e653a70dee3c187b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.721247 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.721193    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.822116 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.822076    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:54.906620 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.906574    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.911203 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:54.911071    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:54.922211 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:54.922191    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.022710 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.022629    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.122916 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.122882    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.166905 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.166872    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:17:55.224333 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.224296    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.226641 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.226625    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 14:17:55.226782 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.226762    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 14:17:55.226836 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.226802    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 14:17:55.298706 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.298618    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 14:12:55 +0000 UTC" deadline="2028-03-11 12:37:31.59188007 +0000 UTC"
Jul 21 14:17:55.298706 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.298656    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14374h19m36.293228629s"
Jul 21 14:17:55.325138 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.325112    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.425224 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.425189    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.450201 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:55.450172    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd975ce7bcfb76736e653a70dee3c187b.slice/crio-9db7490f6810262e611797bf950d8cd73f4be3df342c255a22d191b3bff09540 WatchSource:0}: Error finding container 9db7490f6810262e611797bf950d8cd73f4be3df342c255a22d191b3bff09540: Status 404 returned error can't find the container with id 9db7490f6810262e611797bf950d8cd73f4be3df342c255a22d191b3bff09540
Jul 21 14:17:55.450541 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:55.450526    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775ad56364ad1161b542cba6cf3f9498.slice/crio-fe5c181ff8021c2a100e23c062dbe948e840a796a2a28530931a8e839800801d WatchSource:0}: Error finding container fe5c181ff8021c2a100e23c062dbe948e840a796a2a28530931a8e839800801d: Status 404 returned error can't find the container with id fe5c181ff8021c2a100e23c062dbe948e840a796a2a28530931a8e839800801d
Jul 21 14:17:55.455035 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.455017    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:17:55.465657 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.465610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal" event={"ID":"d975ce7bcfb76736e653a70dee3c187b","Type":"ContainerStarted","Data":"9db7490f6810262e611797bf950d8cd73f4be3df342c255a22d191b3bff09540"}
Jul 21 14:17:55.467002 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.466979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal" event={"ID":"775ad56364ad1161b542cba6cf3f9498","Type":"ContainerStarted","Data":"fe5c181ff8021c2a100e23c062dbe948e840a796a2a28530931a8e839800801d"}
Jul 21 14:17:55.525612 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.525569    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.627748 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.627668    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.729807 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:55.729765    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-87.ec2.internal\" not found"
Jul 21 14:17:55.741746 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.741726    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:17:55.794402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.794373    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:55.807087 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.807070    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 14:17:55.810302 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.810288    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal"
Jul 21 14:17:55.819742 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.819721    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 14:17:55.877216 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:55.877189    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:17:56.272157 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.272118    2569 apiserver.go:52] "Watching apiserver"
Jul 21 14:17:56.277488 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.277442    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 14:17:56.277955 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.277926    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-n99kh","openshift-image-registry/node-ca-7bvzz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal","openshift-network-diagnostics/network-check-target-x296g","openshift-network-operator/iptables-alerter-chx6z","kube-system/global-pull-secret-syncer-24x6m","kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal","openshift-multus/multus-additional-cni-plugins-gmhmh","openshift-multus/multus-n7cx2","openshift-multus/network-metrics-daemon-zjckf","openshift-ovn-kubernetes/ovnkube-node-dqzvh","kube-system/konnectivity-agent-zfl49","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l","openshift-cluster-node-tuning-operator/tuned-rr4v6"]
Jul 21 14:17:56.279752 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.279720    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.279868 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.279816    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:17:56.282095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.282072    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.283539 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.283517    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.284574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.284366    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 14:17:56.284574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.284388    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.284574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.284391    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.284574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.284385    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n6k44\""
Jul 21 14:17:56.285156 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.285134    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.285272 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.285250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:17:56.285374 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.285324    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:17:56.285657 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.285631    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.285753 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.285681    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t6swq\""
Jul 21 14:17:56.287275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.287257    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.288683 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.288666    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.288767 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.288745    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:17:56.289381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.289353    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 14:17:56.289381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.289370    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.289568 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.289386    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-889tm\""
Jul 21 14:17:56.289568 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.289354    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.290364 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.290346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.291949 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.291928    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.293010 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.292991    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.293010 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.293005    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 14:17:56.293176 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.293060    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.293384 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.293365    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tkhdk\""
Jul 21 14:17:56.293458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.293407    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.293458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.293437    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 14:17:56.293593 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.293516    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 14:17:56.294116 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.294096    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 14:17:56.294209 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.294144    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 14:17:56.294596 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.294578    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-59mf9\""
Jul 21 14:17:56.295680 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.295655    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 14:17:56.295785 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.295683    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.295785 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.295760    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zsh5h\""
Jul 21 14:17:56.295785 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.295663    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.296333 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.296314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.298284 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.298259    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.299645 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.298753    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sk9kn\""
Jul 21 14:17:56.299645 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.298985    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.299645 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.299379    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.302408 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.301058    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.302408 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.301201    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 14:17:56.302408 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.301822    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 14:17:56.302408 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.302097    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mv2w8\""
Jul 21 14:17:56.302408 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.302362    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 14:17:56.302741 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.302465    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 14:17:56.302741 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.302496    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 14:17:56.303973 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.303955    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 14:17:56.304143 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304128    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 14:17:56.304301 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304286    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w9dtm\""
Jul 21 14:17:56.304381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmbh8\" (UniqueName: \"kubernetes.io/projected/0e335d36-8172-432c-965c-7176028cf41f-kube-api-access-gmbh8\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.304381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304366    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv7s2\" (UniqueName: \"kubernetes.io/projected/3be69bca-279c-4b3f-af6c-c1b478623ade-kube-api-access-kv7s2\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.304499 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304393    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-cnibin\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.304499 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-cnibin\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.304499 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304445    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-host\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.304499 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-iptables-alerter-script\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.304702 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304532    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nfvh\" (UniqueName: \"kubernetes.io/projected/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-kube-api-access-2nfvh\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.304702 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-k8s-cni-cncf-io\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.304702 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-hostroot\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.304702 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304650    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.304702 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304652    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 14:17:56.304900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-os-release\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.304900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.304963 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.304963 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304929    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-etc-selinux\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.304963 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304946    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-sys\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305087 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-var-lib-kubelet\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305087 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.304999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-kubelet-config\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.305087 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-dbus\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.305087 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f035b4e-b8ff-4208-a35d-2727da6e661b-tmp\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305250 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305085    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-kubelet\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.305250 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305115    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-daemon-config\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.305250 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:17:56.305250 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305164    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-socket-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.305250 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-systemd\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-host-slash\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-multus-certs\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305344    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-run\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-tuned\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-netns\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-registration-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.305528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305513    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysctl-d\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305537    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-kubernetes\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbslk\" (UniqueName: \"kubernetes.io/projected/6f035b4e-b8ff-4208-a35d-2727da6e661b-kube-api-access-sbslk\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305585    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysconfig\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305634    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-conf-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvf45\" (UniqueName: \"kubernetes.io/projected/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-kube-api-access-rvf45\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl8k5\" (UniqueName: \"kubernetes.io/projected/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-kube-api-access-zl8k5\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305725    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-system-cni-dir\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhx2\" (UniqueName: \"kubernetes.io/projected/352915b2-a827-439b-9cec-71bfca9b1215-kube-api-access-9xhx2\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305777    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e335d36-8172-432c-965c-7176028cf41f-hosts-file\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4mfv\" (UniqueName: \"kubernetes.io/projected/b5a651ff-94a7-45df-a9a9-b0266bf28223-kube-api-access-p4mfv\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305861    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-modprobe-d\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.305916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305909    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-system-cni-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-cni-binary-copy\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-socket-dir-parent\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.305998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-cni-multus\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5a651ff-94a7-45df-a9a9-b0266bf28223-serviceca\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306043    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-device-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-sys-fs\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-lib-modules\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306110    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-os-release\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a651ff-94a7-45df-a9a9-b0266bf28223-host\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e335d36-8172-432c-965c-7176028cf41f-tmp-dir\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306206    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-cni-bin\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-etc-kubernetes\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysctl-conf\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.306562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.306290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-cni-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.308411 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.308391    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 14:17:56.331976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.331951    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h5fnn"
Jul 21 14:17:56.346823 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.346802    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h5fnn"
Jul 21 14:17:56.395712 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.395682    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 14:17:56.407588 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407559    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4mfv\" (UniqueName: \"kubernetes.io/projected/b5a651ff-94a7-45df-a9a9-b0266bf28223-kube-api-access-p4mfv\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.407709 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407605    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.407709 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-modprobe-d\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.407709 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407657    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-system-cni-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.407709 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407683    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-cni-binary-copy\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.407709 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-socket-dir-parent\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-cni-multus\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407757    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5a651ff-94a7-45df-a9a9-b0266bf28223-serviceca\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-device-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-sys-fs\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-lib-modules\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-os-release\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407887    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-kubelet\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-node-log\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407954    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.407985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.407979    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a651ff-94a7-45df-a9a9-b0266bf28223-host\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e335d36-8172-432c-965c-7176028cf41f-tmp-dir\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-cni-bin\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408059    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-etc-kubernetes\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-cni-bin\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovnkube-config\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca341b8b-89f3-452c-ac8e-a2c40e16bf40-konnectivity-ca\") pod \"konnectivity-agent-zfl49\" (UID: \"ca341b8b-89f3-452c-ac8e-a2c40e16bf40\") " pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysctl-conf\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-cni-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408210    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjqlg\" (UniqueName: \"kubernetes.io/projected/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-kube-api-access-xjqlg\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408235    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmbh8\" (UniqueName: \"kubernetes.io/projected/0e335d36-8172-432c-965c-7176028cf41f-kube-api-access-gmbh8\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv7s2\" (UniqueName: \"kubernetes.io/projected/3be69bca-279c-4b3f-af6c-c1b478623ade-kube-api-access-kv7s2\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-cnibin\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-systemd-units\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-var-lib-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-cnibin\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-host\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.408526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-iptables-alerter-script\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nfvh\" (UniqueName: \"kubernetes.io/projected/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-kube-api-access-2nfvh\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408491    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-k8s-cni-cncf-io\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-hostroot\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408559    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-systemd\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408599    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-os-release\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-etc-selinux\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-sys\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-var-lib-kubelet\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-kubelet-config\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovn-node-metrics-cert\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-dbus\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408870    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408896    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-log-socket\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.409441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408922    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f035b4e-b8ff-4208-a35d-2727da6e661b-tmp\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-kubelet\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.408975    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-daemon-config\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-slash\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-env-overrides\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovnkube-script-lib\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409088    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-socket-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-systemd\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409175    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-host-slash\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-multus-certs\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-run-netns\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-run\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409321    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-tuned\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-netns\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-etc-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.410234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409408    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-registration-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409436    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysctl-d\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-ovn\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-cni-netd\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409542    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-kubernetes\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbslk\" (UniqueName: \"kubernetes.io/projected/6f035b4e-b8ff-4208-a35d-2727da6e661b-kube-api-access-sbslk\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409630    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409659    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysconfig\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-conf-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409715    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvf45\" (UniqueName: \"kubernetes.io/projected/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-kube-api-access-rvf45\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca341b8b-89f3-452c-ac8e-a2c40e16bf40-agent-certs\") pod \"konnectivity-agent-zfl49\" (UID: \"ca341b8b-89f3-452c-ac8e-a2c40e16bf40\") " pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl8k5\" (UniqueName: \"kubernetes.io/projected/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-kube-api-access-zl8k5\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409805    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-system-cni-dir\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhx2\" (UniqueName: \"kubernetes.io/projected/352915b2-a827-439b-9cec-71bfca9b1215-kube-api-access-9xhx2\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e335d36-8172-432c-965c-7176028cf41f-hosts-file\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.409895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.411109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.410015    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.410751    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.410884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-modprobe-d\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.410897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-sys\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.410974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-dbus\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.410999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-kubelet-config\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.411074    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-system-cni-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.411291    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.411728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-cni-binary-copy\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.411826    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-socket-dir-parent\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.411977 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.411871    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-cni-multus\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.412462 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412259    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-kubelet\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.412462 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412267    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5a651ff-94a7-45df-a9a9-b0266bf28223-serviceca\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.412462 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-device-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.412462 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412423    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-lib-modules\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-os-release\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412552    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-host-slash\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-registration-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysctl-d\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-kubernetes\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412847    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-cnibin\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.412908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-sys-fs\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-daemon-config\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-socket-dir\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-multus-certs\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-cnibin\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413353    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysctl-conf\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-cni-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.413441    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-run\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415127 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.413546    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:17:56.913523635 +0000 UTC m=+3.047919018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413721    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-host\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-multus-conf-dir\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413839    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-systemd\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.413877    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-etc-kubernetes\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414086    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a651ff-94a7-45df-a9a9-b0266bf28223-host\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e335d36-8172-432c-965c-7176028cf41f-tmp-dir\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-iptables-alerter-script\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414319    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-netns\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-var-lib-cni-bin\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e335d36-8172-432c-965c-7176028cf41f-hosts-file\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-var-lib-kubelet\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3be69bca-279c-4b3f-af6c-c1b478623ade-etc-selinux\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-host-run-k8s-cni-cncf-io\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.414596    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414615    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-sysconfig\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414628    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.414696    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:17:56.914678553 +0000 UTC m=+3.049073911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:56.415768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-hostroot\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.416835 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414771    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-os-release\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.416835 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.414853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/352915b2-a827-439b-9cec-71bfca9b1215-system-cni-dir\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.416835 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.415095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/352915b2-a827-439b-9cec-71bfca9b1215-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.416835 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.416624    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f035b4e-b8ff-4208-a35d-2727da6e661b-tmp\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.417309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.417289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f035b4e-b8ff-4208-a35d-2727da6e661b-etc-tuned\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.431321 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.431297    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbslk\" (UniqueName: \"kubernetes.io/projected/6f035b4e-b8ff-4208-a35d-2727da6e661b-kube-api-access-sbslk\") pod \"tuned-rr4v6\" (UID: \"6f035b4e-b8ff-4208-a35d-2727da6e661b\") " pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.431474 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.431358    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmbh8\" (UniqueName: \"kubernetes.io/projected/0e335d36-8172-432c-965c-7176028cf41f-kube-api-access-gmbh8\") pod \"node-resolver-n99kh\" (UID: \"0e335d36-8172-432c-965c-7176028cf41f\") " pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.431692 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.431672    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:17:56.431769 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.431701    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:17:56.431769 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.431715    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:17:56.431769 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.431739    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4mfv\" (UniqueName: \"kubernetes.io/projected/b5a651ff-94a7-45df-a9a9-b0266bf28223-kube-api-access-p4mfv\") pod \"node-ca-7bvzz\" (UID: \"b5a651ff-94a7-45df-a9a9-b0266bf28223\") " pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.431943 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.431786    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:17:56.931761001 +0000 UTC m=+3.066156370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:17:56.435041 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.434995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv7s2\" (UniqueName: \"kubernetes.io/projected/3be69bca-279c-4b3f-af6c-c1b478623ade-kube-api-access-kv7s2\") pod \"aws-ebs-csi-driver-node-gll5l\" (UID: \"3be69bca-279c-4b3f-af6c-c1b478623ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.436346 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.436280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nfvh\" (UniqueName: \"kubernetes.io/projected/55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888-kube-api-access-2nfvh\") pod \"iptables-alerter-chx6z\" (UID: \"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888\") " pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.437789 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.437740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhx2\" (UniqueName: \"kubernetes.io/projected/352915b2-a827-439b-9cec-71bfca9b1215-kube-api-access-9xhx2\") pod \"multus-additional-cni-plugins-gmhmh\" (UID: \"352915b2-a827-439b-9cec-71bfca9b1215\") " pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.438337 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.438316    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl8k5\" (UniqueName: \"kubernetes.io/projected/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-kube-api-access-zl8k5\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.440609 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.440584    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvf45\" (UniqueName: \"kubernetes.io/projected/8bc1f0b1-fead-4b4a-a458-f744e8695fa2-kube-api-access-rvf45\") pod \"multus-n7cx2\" (UID: \"8bc1f0b1-fead-4b4a-a458-f744e8695fa2\") " pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.511216 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511176    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovn-node-metrics-cert\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511248    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-log-socket\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-slash\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-env-overrides\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511328    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovnkube-script-lib\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-run-netns\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-log-socket\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-etc-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-ovn\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-cni-netd\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511573    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca341b8b-89f3-452c-ac8e-a2c40e16bf40-agent-certs\") pod \"konnectivity-agent-zfl49\" (UID: \"ca341b8b-89f3-452c-ac8e-a2c40e16bf40\") " pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-kubelet\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511683    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-node-log\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-ovn\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-cni-bin\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-run-netns\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovnkube-config\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca341b8b-89f3-452c-ac8e-a2c40e16bf40-konnectivity-ca\") pod \"konnectivity-agent-zfl49\" (UID: \"ca341b8b-89f3-452c-ac8e-a2c40e16bf40\") " pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjqlg\" (UniqueName: \"kubernetes.io/projected/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-kube-api-access-xjqlg\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.511856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-systemd-units\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-var-lib-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.511926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-systemd\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-run-systemd\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovnkube-script-lib\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-etc-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512104    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-slash\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512268    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-systemd-units\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-var-lib-openvswitch\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-kubelet\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-cni-netd\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.512677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512391    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.513196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.513196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-node-log\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.513196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovnkube-config\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.513196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512909    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-host-cni-bin\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.513196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.512982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-env-overrides\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.513436 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.513310    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca341b8b-89f3-452c-ac8e-a2c40e16bf40-konnectivity-ca\") pod \"konnectivity-agent-zfl49\" (UID: \"ca341b8b-89f3-452c-ac8e-a2c40e16bf40\") " pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.515364 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.515339    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-ovn-node-metrics-cert\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.515736 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.515712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca341b8b-89f3-452c-ac8e-a2c40e16bf40-agent-certs\") pod \"konnectivity-agent-zfl49\" (UID: \"ca341b8b-89f3-452c-ac8e-a2c40e16bf40\") " pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.521265 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.521221    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjqlg\" (UniqueName: \"kubernetes.io/projected/9f8e7505-04ac-4106-bc7b-6a7b19d40c7c-kube-api-access-xjqlg\") pod \"ovnkube-node-dqzvh\" (UID: \"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.596630 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.596303    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l"
Jul 21 14:17:56.603776 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.603750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rr4v6"
Jul 21 14:17:56.611207 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.611173    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be69bca_279c_4b3f_af6c_c1b478623ade.slice/crio-8abf2a13a12066b3d040e3a2068cc7c910981c7396882a1ab013b5a48e46aade WatchSource:0}: Error finding container 8abf2a13a12066b3d040e3a2068cc7c910981c7396882a1ab013b5a48e46aade: Status 404 returned error can't find the container with id 8abf2a13a12066b3d040e3a2068cc7c910981c7396882a1ab013b5a48e46aade
Jul 21 14:17:56.613920 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.613898    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-chx6z"
Jul 21 14:17:56.615810 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.615779    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f035b4e_b8ff_4208_a35d_2727da6e661b.slice/crio-d88f8cba34a30b0542d303ea5b12ed367fb3ebdfd9f08b818b8a8d2a04f3d053 WatchSource:0}: Error finding container d88f8cba34a30b0542d303ea5b12ed367fb3ebdfd9f08b818b8a8d2a04f3d053: Status 404 returned error can't find the container with id d88f8cba34a30b0542d303ea5b12ed367fb3ebdfd9f08b818b8a8d2a04f3d053
Jul 21 14:17:56.620361 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.620337    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n7cx2"
Jul 21 14:17:56.624688 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.624663    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cfd8b6_520a_4b59_a5e8_e1ab4d1ab888.slice/crio-1f886f098ecb052c5e97cc8bc3962ae80592adb9f196b1b3422ab46e9ebe9f57 WatchSource:0}: Error finding container 1f886f098ecb052c5e97cc8bc3962ae80592adb9f196b1b3422ab46e9ebe9f57: Status 404 returned error can't find the container with id 1f886f098ecb052c5e97cc8bc3962ae80592adb9f196b1b3422ab46e9ebe9f57
Jul 21 14:17:56.630894 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.630868    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc1f0b1_fead_4b4a_a458_f744e8695fa2.slice/crio-66507e9ccbabaeb792e1ae795b26ecc7935d41f792c5508c8d91045a868e3384 WatchSource:0}: Error finding container 66507e9ccbabaeb792e1ae795b26ecc7935d41f792c5508c8d91045a868e3384: Status 404 returned error can't find the container with id 66507e9ccbabaeb792e1ae795b26ecc7935d41f792c5508c8d91045a868e3384
Jul 21 14:17:56.636271 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.636247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmhmh"
Jul 21 14:17:56.643076 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.643022    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7bvzz"
Jul 21 14:17:56.646258 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.646232    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352915b2_a827_439b_9cec_71bfca9b1215.slice/crio-1ffb9214345bb4051ebd8e256bd00abeda1c51244d03ee48703d5cb7622e0769 WatchSource:0}: Error finding container 1ffb9214345bb4051ebd8e256bd00abeda1c51244d03ee48703d5cb7622e0769: Status 404 returned error can't find the container with id 1ffb9214345bb4051ebd8e256bd00abeda1c51244d03ee48703d5cb7622e0769
Jul 21 14:17:56.650927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.650854    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n99kh"
Jul 21 14:17:56.653742 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.653718    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a651ff_94a7_45df_a9a9_b0266bf28223.slice/crio-0bc05a6ed5c48f88d0261105058083d479a2e25e444ace0dbf10a3060e1014c3 WatchSource:0}: Error finding container 0bc05a6ed5c48f88d0261105058083d479a2e25e444ace0dbf10a3060e1014c3: Status 404 returned error can't find the container with id 0bc05a6ed5c48f88d0261105058083d479a2e25e444ace0dbf10a3060e1014c3
Jul 21 14:17:56.660110 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.660091    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:17:56.661948 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.661927    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e335d36_8172_432c_965c_7176028cf41f.slice/crio-aad0c45164b45abdc116e2afab0de9456f82c93348ab16690f9862f50e38c074 WatchSource:0}: Error finding container aad0c45164b45abdc116e2afab0de9456f82c93348ab16690f9862f50e38c074: Status 404 returned error can't find the container with id aad0c45164b45abdc116e2afab0de9456f82c93348ab16690f9862f50e38c074
Jul 21 14:17:56.668318 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.668295    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:17:56.669801 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.669778    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8e7505_04ac_4106_bc7b_6a7b19d40c7c.slice/crio-a19c81ac0d23a4e16e322953fa2add4cf9d3930782a5cdde5eab9146552f16db WatchSource:0}: Error finding container a19c81ac0d23a4e16e322953fa2add4cf9d3930782a5cdde5eab9146552f16db: Status 404 returned error can't find the container with id a19c81ac0d23a4e16e322953fa2add4cf9d3930782a5cdde5eab9146552f16db
Jul 21 14:17:56.680242 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:17:56.679979    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca341b8b_89f3_452c_ac8e_a2c40e16bf40.slice/crio-879add8aacc0e929b1c87f732e885c534ce314a7cee3c0586be85759a1e8172d WatchSource:0}: Error finding container 879add8aacc0e929b1c87f732e885c534ce314a7cee3c0586be85759a1e8172d: Status 404 returned error can't find the container with id 879add8aacc0e929b1c87f732e885c534ce314a7cee3c0586be85759a1e8172d
Jul 21 14:17:56.680242 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.680072    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:17:56.857750 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.857674    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:17:56.915080 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.915041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:56.915335 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:56.915115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:56.915335 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.915220    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:56.915335 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.915234    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:56.915335 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.915295    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:17:57.915278934 +0000 UTC m=+4.049674288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:56.915335 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:56.915314    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:17:57.915304659 +0000 UTC m=+4.049700016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:57.016132 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.016088    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:17:57.016302 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.016221    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:17:57.016302 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.016249    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:17:57.016302 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.016262    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:17:57.016420 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.016324    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:17:58.016307773 +0000 UTC m=+4.150703126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:17:57.348874 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.348832    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 14:12:56 +0000 UTC" deadline="2028-02-23 07:08:51.858558652 +0000 UTC"
Jul 21 14:17:57.348874 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.348870    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13960h50m54.509694204s"
Jul 21 14:17:57.471316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.471226    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zfl49" event={"ID":"ca341b8b-89f3-452c-ac8e-a2c40e16bf40","Type":"ContainerStarted","Data":"879add8aacc0e929b1c87f732e885c534ce314a7cee3c0586be85759a1e8172d"}
Jul 21 14:17:57.472646 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.472614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n99kh" event={"ID":"0e335d36-8172-432c-965c-7176028cf41f","Type":"ContainerStarted","Data":"aad0c45164b45abdc116e2afab0de9456f82c93348ab16690f9862f50e38c074"}
Jul 21 14:17:57.473757 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.473714    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7bvzz" event={"ID":"b5a651ff-94a7-45df-a9a9-b0266bf28223","Type":"ContainerStarted","Data":"0bc05a6ed5c48f88d0261105058083d479a2e25e444ace0dbf10a3060e1014c3"}
Jul 21 14:17:57.474930 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.474904    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n7cx2" event={"ID":"8bc1f0b1-fead-4b4a-a458-f744e8695fa2","Type":"ContainerStarted","Data":"66507e9ccbabaeb792e1ae795b26ecc7935d41f792c5508c8d91045a868e3384"}
Jul 21 14:17:57.476154 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.476129    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"a19c81ac0d23a4e16e322953fa2add4cf9d3930782a5cdde5eab9146552f16db"}
Jul 21 14:17:57.477317 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.477295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerStarted","Data":"1ffb9214345bb4051ebd8e256bd00abeda1c51244d03ee48703d5cb7622e0769"}
Jul 21 14:17:57.478486 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.478447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-chx6z" event={"ID":"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888","Type":"ContainerStarted","Data":"1f886f098ecb052c5e97cc8bc3962ae80592adb9f196b1b3422ab46e9ebe9f57"}
Jul 21 14:17:57.479523 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.479503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rr4v6" event={"ID":"6f035b4e-b8ff-4208-a35d-2727da6e661b","Type":"ContainerStarted","Data":"d88f8cba34a30b0542d303ea5b12ed367fb3ebdfd9f08b818b8a8d2a04f3d053"}
Jul 21 14:17:57.480710 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.480692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l" event={"ID":"3be69bca-279c-4b3f-af6c-c1b478623ade","Type":"ContainerStarted","Data":"8abf2a13a12066b3d040e3a2068cc7c910981c7396882a1ab013b5a48e46aade"}
Jul 21 14:17:57.924472 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.924360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:57.924636 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:57.924470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:57.924636 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.924615    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:57.924764 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.924685    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:17:59.924663719 +0000 UTC m=+6.059059077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:57.925188 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.925166    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:57.925286 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:57.925226    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:17:59.925207834 +0000 UTC m=+6.059603187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:58.025236 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.025168    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:17:58.025412 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.025357    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:17:58.025412 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.025380    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:17:58.025412 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.025393    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:17:58.025625 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.025478    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:00.025435883 +0000 UTC m=+6.159831240 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:17:58.349483 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.349227    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 14:12:56 +0000 UTC" deadline="2028-04-19 00:23:40.07019415 +0000 UTC"
Jul 21 14:17:58.349483 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.349486    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15298h5m41.720716023s"
Jul 21 14:17:58.464717 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.464681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:58.464940 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.464825    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:17:58.465274 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.465243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:58.465376 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.465355    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:17:58.465833 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.465812    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:17:58.465956 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:58.465913    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:17:58.491686 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:58.491050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal" event={"ID":"775ad56364ad1161b542cba6cf3f9498","Type":"ContainerStarted","Data":"cfe1b6dd8c5d38e3f6de0ca1f8a506e9a45c79a1e695a2fb2b4095af07c335c1"}
Jul 21 14:17:59.502121 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:59.502082    2569 generic.go:356] "Generic (PLEG): container finished" podID="d975ce7bcfb76736e653a70dee3c187b" containerID="15ebdf09a5bec846de93913a3101fb2f27d61780207e97d90da91e9dfff13820" exitCode=0
Jul 21 14:17:59.502699 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:59.502677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal" event={"ID":"d975ce7bcfb76736e653a70dee3c187b","Type":"ContainerDied","Data":"15ebdf09a5bec846de93913a3101fb2f27d61780207e97d90da91e9dfff13820"}
Jul 21 14:17:59.520680 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:59.520623    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-87.ec2.internal" podStartSLOduration=3.520601896 podStartE2EDuration="3.520601896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:17:58.51181665 +0000 UTC m=+4.646212030" watchObservedRunningTime="2026-07-21 14:17:59.520601896 +0000 UTC m=+5.654997275"
Jul 21 14:17:59.941574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:59.941535    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:17:59.941813 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:17:59.941622    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:17:59.941813 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:59.941770    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:59.941967 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:59.941841    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:03.941821048 +0000 UTC m=+10.076216423 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:17:59.942281 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:59.942259    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:17:59.942387 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:17:59.942328    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:18:03.942311926 +0000 UTC m=+10.076707278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:00.042790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:00.042468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:00.042790 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.042659    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:18:00.042790 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.042682    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:18:00.042790 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.042696    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:00.042790 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.042768    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:04.042746773 +0000 UTC m=+10.177142133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:00.464083 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:00.463547    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:00.464083 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:00.463548    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:00.464083 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.463712    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:00.464083 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.463840    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:00.464083 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:00.463888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:00.464083 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:00.463949    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:02.463203 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:02.462632    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:02.463203 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:02.462664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:02.463203 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:02.462791    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:02.463203 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:02.462859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:02.463203 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:02.462958    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:02.463203 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:02.463081    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:03.976497 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:03.976439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:03.977011 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:03.976556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:03.977011 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:03.976704    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:03.977011 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:03.976774    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:18:11.976752207 +0000 UTC m=+18.111147563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:03.977201 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:03.977107    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:18:03.977201 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:03.977163    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:11.977145768 +0000 UTC m=+18.111541122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:18:04.076954 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:04.076913    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:04.077157 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.077130    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:18:04.077157 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.077160    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:18:04.077361 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.077171    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:04.077361 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.077219    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:12.077204137 +0000 UTC m=+18.211599491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:04.463966 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:04.463462    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:04.463966 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:04.463470    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:04.463966 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.463603    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:04.463966 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:04.463476    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:04.463966 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.463682    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:04.463966 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:04.463775    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:06.465305 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:06.464706    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:06.465305 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:06.464818    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:06.465305 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:06.465108    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:06.465305 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:06.465196    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:06.465305 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:06.465217    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:06.465305 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:06.465278    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:08.462153 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:08.462119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:08.462632 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:08.462276    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:08.462632 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:08.462294    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:08.462632 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:08.462362    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:08.462632 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:08.462457    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:08.462632 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:08.462526    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:10.463031 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:10.462994    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:10.463530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:10.462997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:10.463530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:10.463133    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:10.463530 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:10.463127    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:10.463530 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:10.463265    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:10.463530 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:10.463375    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:12.035605 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:12.035572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:12.036025 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:12.035622    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:12.036025 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.035714    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:12.036025 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.035717    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:18:12.036025 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.035769    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:18:28.035755359 +0000 UTC m=+34.170150713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:12.036025 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.035783    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:28.035776339 +0000 UTC m=+34.170171690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:18:12.136066 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:12.136042    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:12.136210 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.136196    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:18:12.136247 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.136216    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:18:12.136247 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.136225    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:12.136310 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.136278    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:18:28.136260455 +0000 UTC m=+34.270655808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:12.462282 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:12.462245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:12.462468 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:12.462245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:12.462468 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.462386    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:12.462590 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.462487    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:12.462590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:12.462246    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:12.462684 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:12.462601    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:14.463968 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:14.463935    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:14.464538 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:14.464025    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:14.464538 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:14.464047    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:14.464538 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:14.464044    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:14.464538 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:14.464128    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:14.464538 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:14.464198    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:16.463078 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:16.463046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:16.463509 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:16.463083    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:16.463509 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:16.463184    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:16.463509 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:16.463229    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:16.463509 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:16.463282    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:16.463509 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:16.463392    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:17.541492 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.541202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rr4v6" event={"ID":"6f035b4e-b8ff-4208-a35d-2727da6e661b","Type":"ContainerStarted","Data":"6ed756f9714263cbde5bddc033f82ca173d559cb14b6900b34d6e49114f57a47"}
Jul 21 14:18:17.542674 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.542635    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l" event={"ID":"3be69bca-279c-4b3f-af6c-c1b478623ade","Type":"ContainerStarted","Data":"82d4a62ff3fbda04eb0d723a0e7b4662e51a8e5c128c78cbc3a29968497c8634"}
Jul 21 14:18:17.543896 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.543867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zfl49" event={"ID":"ca341b8b-89f3-452c-ac8e-a2c40e16bf40","Type":"ContainerStarted","Data":"1368cc1b25534b3cb08e1506acf7c32fcd5dcecfdad3ada5913cb4e91889b611"}
Jul 21 14:18:17.545235 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.545211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n99kh" event={"ID":"0e335d36-8172-432c-965c-7176028cf41f","Type":"ContainerStarted","Data":"ebe8bd54363d82a385b369c5e1c547c0e6c9b9c386aab8f9173054ffed0cee08"}
Jul 21 14:18:17.546467 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.546446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7bvzz" event={"ID":"b5a651ff-94a7-45df-a9a9-b0266bf28223","Type":"ContainerStarted","Data":"a59c975cc7b7830b235f3342f750da05cbcaa00e9283f011fcae90971b2568c8"}
Jul 21 14:18:17.547768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.547751    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n7cx2" event={"ID":"8bc1f0b1-fead-4b4a-a458-f744e8695fa2","Type":"ContainerStarted","Data":"7b141362d52fdd2d9a675fb53beb0d83cdc4c317c2f8008170f39976250e9f3a"}
Jul 21 14:18:17.549463 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.549444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal" event={"ID":"d975ce7bcfb76736e653a70dee3c187b","Type":"ContainerStarted","Data":"b5d31913faf49c553823e32a46139ee2387eabd02dbb5fe9741777341607b750"}
Jul 21 14:18:17.552138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.552119    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"f5e8922e095779609b054d48237911d1362d4378b730390dc6f896ca7ed50278"}
Jul 21 14:18:17.552211 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.552171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"d9c3beb83b584d93c2fe592cbed0489a7b34d8be2c22875847a831f8fdac1361"}
Jul 21 14:18:17.552211 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.552201    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"6351abb2b7ad794d13f8ac972cfea713c091e0d05031caf714172f46cd55f31d"}
Jul 21 14:18:17.552285 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.552235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"01207bbaaf1f84a4e674ae133b2ad83df3b80577c196831be4b56a5988a88ce9"}
Jul 21 14:18:17.552285 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.552269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"e9b6cf10b2f3ba88011db59354247cfb37ae399b453b173b660833ff2af341c4"}
Jul 21 14:18:17.552366 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.552314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"4b1a6f5cfb185021dfae42a5a44c9d5ec272e883d0fef787fde22ab1e7cc8447"}
Jul 21 14:18:17.553447 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.553411    2569 generic.go:356] "Generic (PLEG): container finished" podID="352915b2-a827-439b-9cec-71bfca9b1215" containerID="8b8afdda1d58cb222739ae9e3fc5fd7751554dd03082d2b418a2724d440ffb78" exitCode=0
Jul 21 14:18:17.553522 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.553459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerDied","Data":"8b8afdda1d58cb222739ae9e3fc5fd7751554dd03082d2b418a2724d440ffb78"}
Jul 21 14:18:17.562646 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.562602    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rr4v6" podStartSLOduration=2.553183772 podStartE2EDuration="22.562588586s" totalImagesPullingTime="20.009404814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:17.562378625 +0000 UTC m=+23.696774000" watchObservedRunningTime="2026-07-21 14:18:17.562588586 +0000 UTC m=+23.696983988"
Jul 21 14:18:17.594650 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.594593    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n99kh" podStartSLOduration=2.628823883 podStartE2EDuration="22.594573809s" totalImagesPullingTime="19.965749926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:17.594232579 +0000 UTC m=+23.728627975" watchObservedRunningTime="2026-07-21 14:18:17.594573809 +0000 UTC m=+23.728969185"
Jul 21 14:18:17.594792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.594721    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7bvzz" podStartSLOduration=2.626909929 podStartE2EDuration="22.594685609s" totalImagesPullingTime="19.96777568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:17.578501327 +0000 UTC m=+23.712896700" watchObservedRunningTime="2026-07-21 14:18:17.594685609 +0000 UTC m=+23.729080984"
Jul 21 14:18:17.611002 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.610944    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zfl49" podStartSLOduration=7.151560036 podStartE2EDuration="22.610930357s" totalImagesPullingTime="15.459370321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:17.610556002 +0000 UTC m=+23.744951375" watchObservedRunningTime="2026-07-21 14:18:17.610930357 +0000 UTC m=+23.745325867"
Jul 21 14:18:17.628283 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.628237    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-87.ec2.internal" podStartSLOduration=21.628218881 podStartE2EDuration="21.628218881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:17.627716096 +0000 UTC m=+23.762111469" watchObservedRunningTime="2026-07-21 14:18:17.628218881 +0000 UTC m=+23.762614254"
Jul 21 14:18:17.648958 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:17.648901    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n7cx2" podStartSLOduration=2.584767723 podStartE2EDuration="22.648886019s" totalImagesPullingTime="20.064118296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:17.64834224 +0000 UTC m=+23.782737638" watchObservedRunningTime="2026-07-21 14:18:17.648886019 +0000 UTC m=+23.783281393"
Jul 21 14:18:18.254431 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.254376    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 14:18:18.376946 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.376831    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T14:18:18.254413821Z","UUID":"c47c64ec-d671-41ff-9c66-cdf5e20d022c","Handler":null,"Name":"","Endpoint":""}
Jul 21 14:18:18.378406 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.378381    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 14:18:18.378552 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.378440    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 14:18:18.463119 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.463075    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:18.463119 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.463096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:18.463332 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.463096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:18.463332 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:18.463214    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:18.463458 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:18.463363    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:18.463515 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:18.463473    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:18.557927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.557888    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l" event={"ID":"3be69bca-279c-4b3f-af6c-c1b478623ade","Type":"ContainerStarted","Data":"b453eb64322f9e595b9a6e65d89fa06403174263847965f92af8f1938b42533a"}
Jul 21 14:18:18.559472 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.559442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-chx6z" event={"ID":"55cfd8b6-520a-4b59-a5e8-e1ab4d1ab888","Type":"ContainerStarted","Data":"49a6e8197c8e84dc8dee946651e19baf3acb12b11faf9fb8cacd97865eb51cef"}
Jul 21 14:18:18.578356 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:18.578307    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-chx6z" podStartSLOduration=3.5808642219999998 podStartE2EDuration="23.578290458s" totalImagesPullingTime="19.997426236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:18.577772555 +0000 UTC m=+24.712167928" watchObservedRunningTime="2026-07-21 14:18:18.578290458 +0000 UTC m=+24.712685832"
Jul 21 14:18:19.564824 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:19.564747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l" event={"ID":"3be69bca-279c-4b3f-af6c-c1b478623ade","Type":"ContainerStarted","Data":"17ce22ec2af453afef0c5fea38ff3224f3c9fc7840fc632119cfe0bbe6f73452"}
Jul 21 14:18:19.573266 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:19.573229    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"f0e5b13e63bb6f0e306453f9cf626b02f209a553456282ccd68b96e95300d677"}
Jul 21 14:18:19.586298 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:19.586255    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gll5l" podStartSLOduration=2.330333048 podStartE2EDuration="24.586242798s" totalImagesPullingTime="22.25590975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:19.586115076 +0000 UTC m=+25.720510466" watchObservedRunningTime="2026-07-21 14:18:19.586242798 +0000 UTC m=+25.720638173"
Jul 21 14:18:20.462379 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:20.462351    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:20.462568 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:20.462387    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:20.462568 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:20.462351    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:20.462568 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:20.462470    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:20.462730 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:20.462577    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:20.462730 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:20.462635    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:21.747488 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:21.747197    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:18:21.748014 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:21.747994    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:18:21.748270 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:21.748240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:18:21.748596 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:21.748580    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zfl49"
Jul 21 14:18:22.463053 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.462844    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:22.463197 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.462943    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:22.463197 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:22.463148    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:22.463290 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:22.463211    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:22.463290 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.462969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:22.463351 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:22.463319    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:22.583460 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.583354    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" event={"ID":"9f8e7505-04ac-4106-bc7b-6a7b19d40c7c","Type":"ContainerStarted","Data":"d065b22090b3e49db2de6fb79ec66a682b278a3669b95c7dea654b23ff1d0605"}
Jul 21 14:18:22.583797 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.583706    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:18:22.585313 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.585289    2569 generic.go:356] "Generic (PLEG): container finished" podID="352915b2-a827-439b-9cec-71bfca9b1215" containerID="c943468f0a5b6cdc24554ee7ad94e8a328777e3d63092092f9c1205c33c9621f" exitCode=0
Jul 21 14:18:22.585397 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.585370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerDied","Data":"c943468f0a5b6cdc24554ee7ad94e8a328777e3d63092092f9c1205c33c9621f"}
Jul 21 14:18:22.600634 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.600612    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:18:22.623142 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:22.623100    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh" podStartSLOduration=7.668171277 podStartE2EDuration="27.6230857s" totalImagesPullingTime="19.954914423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:22.622133382 +0000 UTC m=+28.756528756" watchObservedRunningTime="2026-07-21 14:18:22.6230857 +0000 UTC m=+28.757481074"
Jul 21 14:18:23.588131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:23.588094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:18:23.588937 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:23.588552    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:18:23.603670 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:23.603651    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:18:24.463590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:24.463557    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:24.463775 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:24.463630    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:24.463775 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:24.463642    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:24.463775 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:24.463730    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:24.463935 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:24.463784    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:24.463935 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:24.463829    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:24.592328 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:24.592297    2569 generic.go:356] "Generic (PLEG): container finished" podID="352915b2-a827-439b-9cec-71bfca9b1215" containerID="0bb48e8ea49d7780c559f7481fc49fe2c900b4540570f867d640741fb7002d79" exitCode=0
Jul 21 14:18:24.592667 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:24.592378    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerDied","Data":"0bb48e8ea49d7780c559f7481fc49fe2c900b4540570f867d640741fb7002d79"}
Jul 21 14:18:26.462787 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:26.462755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:26.463258 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:26.462755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:26.463258 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:26.462867    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:26.463258 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:26.462951    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:26.463258 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:26.463003    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:26.463258 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:26.463055    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:28.062883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.062846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:28.063441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.062910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:28.063441 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.063033    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:18:28.063441 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.063034    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:28.063441 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.063107    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs podName:9e23ecc9-b2ec-4ce2-b581-6b00e8314017 nodeName:}" failed. No retries permitted until 2026-07-21 14:19:00.063086399 +0000 UTC m=+66.197481751 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs") pod "network-metrics-daemon-zjckf" (UID: "9e23ecc9-b2ec-4ce2-b581-6b00e8314017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:18:28.063441 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.063126    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret podName:55a3aa06-f532-4817-a3ad-dfbcf037ec5a nodeName:}" failed. No retries permitted until 2026-07-21 14:19:00.063116301 +0000 UTC m=+66.197511654 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret") pod "global-pull-secret-syncer-24x6m" (UID: "55a3aa06-f532-4817-a3ad-dfbcf037ec5a") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:18:28.163837 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.163801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:28.163989 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.163959    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:18:28.163989 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.163978    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:18:28.163989 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.163987    2569 projected.go:196] Error preparing data for projected volume kube-api-access-dfdzz for pod openshift-network-diagnostics/network-check-target-x296g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:28.164085 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.164041    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz podName:a52903c5-7ce8-423b-aeb9-2ee6079753d6 nodeName:}" failed. No retries permitted until 2026-07-21 14:19:00.164026056 +0000 UTC m=+66.298421409 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dfdzz" (UniqueName: "kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz") pod "network-check-target-x296g" (UID: "a52903c5-7ce8-423b-aeb9-2ee6079753d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:18:28.463199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.463163    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:28.463358 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.463163    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:28.463358 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.463272    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:28.463478 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.463177    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:28.463478 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.463364    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:28.463550 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:28.463481    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:28.602717 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.602683    2569 generic.go:356] "Generic (PLEG): container finished" podID="352915b2-a827-439b-9cec-71bfca9b1215" containerID="98631fd3684fb3cdfc74d357f7d70d46aa6f5d5864ba15d26899a0002ac471dc" exitCode=0
Jul 21 14:18:28.602877 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:28.602726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerDied","Data":"98631fd3684fb3cdfc74d357f7d70d46aa6f5d5864ba15d26899a0002ac471dc"}
Jul 21 14:18:30.201187 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:30.201154    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24x6m"]
Jul 21 14:18:30.201907 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:30.201254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:30.201907 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:30.201369    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:30.206163 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:30.204400    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjckf"]
Jul 21 14:18:30.206163 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:30.204529    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:30.206163 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:30.204639    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:30.206163 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:30.205626    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x296g"]
Jul 21 14:18:30.206163 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:30.205702    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:30.206163 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:30.205770    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:31.462462 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:31.462394    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:31.462853 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:31.462437    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:31.462853 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:31.462572    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:31.462853 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:31.462711    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:32.462884 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:32.462854    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:32.463310 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:32.462989    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:33.462886 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:33.462856    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:33.463071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:33.462860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:33.463071 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:33.462952    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24x6m" podUID="55a3aa06-f532-4817-a3ad-dfbcf037ec5a"
Jul 21 14:18:33.463071 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:33.463047    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjckf" podUID="9e23ecc9-b2ec-4ce2-b581-6b00e8314017"
Jul 21 14:18:33.616766 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:33.616736    2569 generic.go:356] "Generic (PLEG): container finished" podID="352915b2-a827-439b-9cec-71bfca9b1215" containerID="d6e1243e77e921156d0daaf38ee8e467187a9b9eeacf53087023fdca98c81571" exitCode=0
Jul 21 14:18:33.616917 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:33.616780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerDied","Data":"d6e1243e77e921156d0daaf38ee8e467187a9b9eeacf53087023fdca98c81571"}
Jul 21 14:18:34.463915 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.463884    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:34.464332 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:34.463967    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x296g" podUID="a52903c5-7ce8-423b-aeb9-2ee6079753d6"
Jul 21 14:18:34.621728 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.621696    2569 generic.go:356] "Generic (PLEG): container finished" podID="352915b2-a827-439b-9cec-71bfca9b1215" containerID="b2ec134526061e9dc7777791e74ab8ca63f77dda22b7a08517af4e62c38328b3" exitCode=0
Jul 21 14:18:34.621881 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.621733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerDied","Data":"b2ec134526061e9dc7777791e74ab8ca63f77dda22b7a08517af4e62c38328b3"}
Jul 21 14:18:34.670039 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.669931    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-87.ec2.internal" event="NodeReady"
Jul 21 14:18:34.670110 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.670047    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 14:18:34.707000 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.706974    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5888557764-m6s9h"]
Jul 21 14:18:34.732154 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.732133    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m4ftr"]
Jul 21 14:18:34.732287 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.732254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.734590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.734386    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2s4f2\""
Jul 21 14:18:34.734590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.734470    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 14:18:34.734590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.734511    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 14:18:34.734590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.734554    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 14:18:34.746020 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.745997    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 14:18:34.750386 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.750369    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zm5zn"]
Jul 21 14:18:34.750511 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.750495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:34.752286 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.752268    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rkkbj\""
Jul 21 14:18:34.752286 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.752296    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 14:18:34.752487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.752444    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 14:18:34.766685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.766519    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5888557764-m6s9h"]
Jul 21 14:18:34.766685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.766557    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zm5zn"]
Jul 21 14:18:34.766685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.766576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4ftr"]
Jul 21 14:18:34.766685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.766609    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:34.769895 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.769873    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 14:18:34.770024 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.769899    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 14:18:34.770134 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.770117    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s5hdl\""
Jul 21 14:18:34.770209 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.770191    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 14:18:34.912243 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912215    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-installation-pull-secrets\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912409 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-registry-certificates\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912409 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-image-registry-private-configuration\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912409 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912329    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-bound-sa-token\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912409 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpln9\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-kube-api-access-jpln9\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9918107e-39c4-411c-99f2-c7ebe307cb65-config-volume\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9918107e-39c4-411c-99f2-c7ebe307cb65-tmp-dir\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-ca-trust-extracted\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-trusted-ca\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912549    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85ce1fac-29e6-46c0-8aed-8a6f33357deb-cert\") pod \"ingress-canary-zm5zn\" (UID: \"85ce1fac-29e6-46c0-8aed-8a6f33357deb\") " pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv4ng\" (UniqueName: \"kubernetes.io/projected/9918107e-39c4-411c-99f2-c7ebe307cb65-kube-api-access-zv4ng\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9918107e-39c4-411c-99f2-c7ebe307cb65-metrics-tls\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:34.912622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czp7j\" (UniqueName: \"kubernetes.io/projected/85ce1fac-29e6-46c0-8aed-8a6f33357deb-kube-api-access-czp7j\") pod \"ingress-canary-zm5zn\" (UID: \"85ce1fac-29e6-46c0-8aed-8a6f33357deb\") " pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:34.912866 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:34.912651    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-registry-tls\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013220 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013135    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-registry-certificates\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013220 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-image-registry-private-configuration\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013220 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-bound-sa-token\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013220 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpln9\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-kube-api-access-jpln9\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013220 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9918107e-39c4-411c-99f2-c7ebe307cb65-config-volume\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9918107e-39c4-411c-99f2-c7ebe307cb65-tmp-dir\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013248    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-ca-trust-extracted\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-trusted-ca\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85ce1fac-29e6-46c0-8aed-8a6f33357deb-cert\") pod \"ingress-canary-zm5zn\" (UID: \"85ce1fac-29e6-46c0-8aed-8a6f33357deb\") " pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4ng\" (UniqueName: \"kubernetes.io/projected/9918107e-39c4-411c-99f2-c7ebe307cb65-kube-api-access-zv4ng\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9918107e-39c4-411c-99f2-c7ebe307cb65-metrics-tls\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013381    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czp7j\" (UniqueName: \"kubernetes.io/projected/85ce1fac-29e6-46c0-8aed-8a6f33357deb-kube-api-access-czp7j\") pod \"ingress-canary-zm5zn\" (UID: \"85ce1fac-29e6-46c0-8aed-8a6f33357deb\") " pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:35.013618 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013608    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-registry-tls\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-installation-pull-secrets\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013699    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9918107e-39c4-411c-99f2-c7ebe307cb65-tmp-dir\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.013985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-ca-trust-extracted\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.013985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.013963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9918107e-39c4-411c-99f2-c7ebe307cb65-config-volume\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.014166 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.014135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-registry-certificates\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.014334 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.014293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-trusted-ca\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.017779 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.017757    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-installation-pull-secrets\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.017869 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.017800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9918107e-39c4-411c-99f2-c7ebe307cb65-metrics-tls\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.017869 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.017809    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-image-registry-private-configuration\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.017940 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.017885    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-registry-tls\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.018005 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.017986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85ce1fac-29e6-46c0-8aed-8a6f33357deb-cert\") pod \"ingress-canary-zm5zn\" (UID: \"85ce1fac-29e6-46c0-8aed-8a6f33357deb\") " pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:35.023928 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.023895    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-bound-sa-token\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.024214 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.024162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpln9\" (UniqueName: \"kubernetes.io/projected/062b0576-1508-4eed-b37b-cb1f0b1ae2b1-kube-api-access-jpln9\") pod \"image-registry-5888557764-m6s9h\" (UID: \"062b0576-1508-4eed-b37b-cb1f0b1ae2b1\") " pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.024214 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.024194    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czp7j\" (UniqueName: \"kubernetes.io/projected/85ce1fac-29e6-46c0-8aed-8a6f33357deb-kube-api-access-czp7j\") pod \"ingress-canary-zm5zn\" (UID: \"85ce1fac-29e6-46c0-8aed-8a6f33357deb\") " pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:35.024620 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.024590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv4ng\" (UniqueName: \"kubernetes.io/projected/9918107e-39c4-411c-99f2-c7ebe307cb65-kube-api-access-zv4ng\") pod \"dns-default-m4ftr\" (UID: \"9918107e-39c4-411c-99f2-c7ebe307cb65\") " pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.043517 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.043493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.058161 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.058139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:35.074879 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.074857    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zm5zn"
Jul 21 14:18:35.254677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.254633    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4ftr"]
Jul 21 14:18:35.258687 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:18:35.258652    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9918107e_39c4_411c_99f2_c7ebe307cb65.slice/crio-4a4f50da3ad61cdf68d0d73a774c44f815557a5cbf7c54bb0c729ddb7f88d891 WatchSource:0}: Error finding container 4a4f50da3ad61cdf68d0d73a774c44f815557a5cbf7c54bb0c729ddb7f88d891: Status 404 returned error can't find the container with id 4a4f50da3ad61cdf68d0d73a774c44f815557a5cbf7c54bb0c729ddb7f88d891
Jul 21 14:18:35.263758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.263704    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5888557764-m6s9h"]
Jul 21 14:18:35.268303 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:18:35.268277    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062b0576_1508_4eed_b37b_cb1f0b1ae2b1.slice/crio-47598812389ebecbd607bfec2bf4465a13ecc7a115d7149704ccad8a5a9f998b WatchSource:0}: Error finding container 47598812389ebecbd607bfec2bf4465a13ecc7a115d7149704ccad8a5a9f998b: Status 404 returned error can't find the container with id 47598812389ebecbd607bfec2bf4465a13ecc7a115d7149704ccad8a5a9f998b
Jul 21 14:18:35.275054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.275026    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zm5zn"]
Jul 21 14:18:35.278499 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:18:35.278476    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ce1fac_29e6_46c0_8aed_8a6f33357deb.slice/crio-32e2cd08deb2d1768a99c7e5da2683e0f4c072e185eaf2611ce89f5a6acfb4c4 WatchSource:0}: Error finding container 32e2cd08deb2d1768a99c7e5da2683e0f4c072e185eaf2611ce89f5a6acfb4c4: Status 404 returned error can't find the container with id 32e2cd08deb2d1768a99c7e5da2683e0f4c072e185eaf2611ce89f5a6acfb4c4
Jul 21 14:18:35.462516 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.462484    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:18:35.462686 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.462484    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:18:35.464959 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.464932    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 14:18:35.464959 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.464954    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2kbnq\""
Jul 21 14:18:35.465357 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.465041    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 14:18:35.625799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.625701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5888557764-m6s9h" event={"ID":"062b0576-1508-4eed-b37b-cb1f0b1ae2b1","Type":"ContainerStarted","Data":"53d2e009ae8c02cd5406ac74742089d912ba53494a0f848f2f5114e832e5e646"}
Jul 21 14:18:35.625799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.625760    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5888557764-m6s9h" event={"ID":"062b0576-1508-4eed-b37b-cb1f0b1ae2b1","Type":"ContainerStarted","Data":"47598812389ebecbd607bfec2bf4465a13ecc7a115d7149704ccad8a5a9f998b"}
Jul 21 14:18:35.625799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.625789    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:35.629106 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.629077    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" event={"ID":"352915b2-a827-439b-9cec-71bfca9b1215","Type":"ContainerStarted","Data":"478a55d13159526323853177ef65d634dd977b947584ee3f950ffc6634adda2e"}
Jul 21 14:18:35.630374 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.630353    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zm5zn" event={"ID":"85ce1fac-29e6-46c0-8aed-8a6f33357deb","Type":"ContainerStarted","Data":"32e2cd08deb2d1768a99c7e5da2683e0f4c072e185eaf2611ce89f5a6acfb4c4"}
Jul 21 14:18:35.631483 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.631455    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4ftr" event={"ID":"9918107e-39c4-411c-99f2-c7ebe307cb65","Type":"ContainerStarted","Data":"4a4f50da3ad61cdf68d0d73a774c44f815557a5cbf7c54bb0c729ddb7f88d891"}
Jul 21 14:18:35.647481 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.647443    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5888557764-m6s9h" podStartSLOduration=6.647411312 podStartE2EDuration="6.647411312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:35.647091855 +0000 UTC m=+41.781487230" watchObservedRunningTime="2026-07-21 14:18:35.647411312 +0000 UTC m=+41.781806665"
Jul 21 14:18:35.669802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:35.669756    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gmhmh" podStartSLOduration=7.285439305 podStartE2EDuration="40.669741404s" totalImagesPullingTime="33.384302099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:35.669053948 +0000 UTC m=+41.803449325" watchObservedRunningTime="2026-07-21 14:18:35.669741404 +0000 UTC m=+41.804136772"
Jul 21 14:18:36.464214 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:36.464183    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:18:36.466897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:36.466867    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qcck5\""
Jul 21 14:18:36.467295 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:36.466962    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 14:18:36.467295 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:36.467062    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 14:18:38.640979 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:38.640952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4ftr" event={"ID":"9918107e-39c4-411c-99f2-c7ebe307cb65","Type":"ContainerStarted","Data":"0d56dc8c9240d275c8dcaf93b7fadd21bc73f3c8550eaa6841ef1f2b2a60db99"}
Jul 21 14:18:39.645506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.645459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4ftr" event={"ID":"9918107e-39c4-411c-99f2-c7ebe307cb65","Type":"ContainerStarted","Data":"1de2c49df79cce848fb6b064d5c05af5716b7d172b087043780a60cec20a9f98"}
Jul 21 14:18:39.645921 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.645533    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:39.646904 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.646878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zm5zn" event={"ID":"85ce1fac-29e6-46c0-8aed-8a6f33357deb","Type":"ContainerStarted","Data":"2c5f17217a89eddfee83abd8d6c542cdc866a9c8c7191cc4f7c17534e3b56bd3"}
Jul 21 14:18:39.684693 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.684665    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4plps"]
Jul 21 14:18:39.700122 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.700071    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m4ftr" podStartSLOduration=2.557415514 podStartE2EDuration="5.700053764s" totalImagesPullingTime="3.14263825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:39.698643121 +0000 UTC m=+45.833038495" watchObservedRunningTime="2026-07-21 14:18:39.700053764 +0000 UTC m=+45.834449139"
Jul 21 14:18:39.708583 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.708561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.712479 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.712463    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 14:18:39.713460 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.713441    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4plps"]
Jul 21 14:18:39.716757 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.716736    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 14:18:39.716870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.716781    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 14:18:39.716870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.716789    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m7grf\""
Jul 21 14:18:39.754881 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.754835    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zm5zn" podStartSLOduration=2.338464892 podStartE2EDuration="5.754822969s" totalImagesPullingTime="3.416358077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:39.753871579 +0000 UTC m=+45.888266966" watchObservedRunningTime="2026-07-21 14:18:39.754822969 +0000 UTC m=+45.889218343"
Jul 21 14:18:39.772083 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.772064    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 14:18:39.849435 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.849385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec673038-4e58-4f33-bafe-8155ac3a4efc-data-volume\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.849582 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.849506    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec673038-4e58-4f33-bafe-8155ac3a4efc-crio-socket\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.849582 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.849537    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dvzz\" (UniqueName: \"kubernetes.io/projected/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-api-access-2dvzz\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.849651 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.849586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.849651 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.849641    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.950695 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.950667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec673038-4e58-4f33-bafe-8155ac3a4efc-data-volume\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.950844 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.950714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec673038-4e58-4f33-bafe-8155ac3a4efc-crio-socket\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.950844 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.950740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dvzz\" (UniqueName: \"kubernetes.io/projected/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-api-access-2dvzz\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.950909 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.950858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.950952 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.950897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec673038-4e58-4f33-bafe-8155ac3a4efc-crio-socket\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.950952 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.950936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.952225 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.951373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec673038-4e58-4f33-bafe-8155ac3a4efc-data-volume\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:39.952225 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:39.951401    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 14:18:39.952225 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:39.951560    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm podName:ec673038-4e58-4f33-bafe-8155ac3a4efc nodeName:}" failed. No retries permitted until 2026-07-21 14:18:40.451518048 +0000 UTC m=+46.585913415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4plps" (UID: "ec673038-4e58-4f33-bafe-8155ac3a4efc") : configmap "kube-rbac-proxy" not found
Jul 21 14:18:39.952225 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:39.951589    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:18:39.952524 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:39.952232    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls podName:ec673038-4e58-4f33-bafe-8155ac3a4efc nodeName:}" failed. No retries permitted until 2026-07-21 14:18:40.452210898 +0000 UTC m=+46.586606268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4plps" (UID: "ec673038-4e58-4f33-bafe-8155ac3a4efc") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:18:39.969735 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:39.969703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dvzz\" (UniqueName: \"kubernetes.io/projected/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-api-access-2dvzz\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:40.456065 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:40.456035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:40.456225 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:40.456131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:40.456225 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:40.456204    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:18:40.456225 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:40.456214    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 14:18:40.456344 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:40.456287    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls podName:ec673038-4e58-4f33-bafe-8155ac3a4efc nodeName:}" failed. No retries permitted until 2026-07-21 14:18:41.456254491 +0000 UTC m=+47.590649859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4plps" (UID: "ec673038-4e58-4f33-bafe-8155ac3a4efc") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:18:40.456344 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:40.456304    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm podName:ec673038-4e58-4f33-bafe-8155ac3a4efc nodeName:}" failed. No retries permitted until 2026-07-21 14:18:41.45629342 +0000 UTC m=+47.590688785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4plps" (UID: "ec673038-4e58-4f33-bafe-8155ac3a4efc") : configmap "kube-rbac-proxy" not found
Jul 21 14:18:41.051525 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:41.051502    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4ftr_9918107e-39c4-411c-99f2-c7ebe307cb65/dns/0.log"
Jul 21 14:18:41.230373 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:41.230344    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4ftr_9918107e-39c4-411c-99f2-c7ebe307cb65/kube-rbac-proxy/0.log"
Jul 21 14:18:41.464718 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:41.464682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:41.464903 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:41.464766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:41.464903 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:41.464840    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:18:41.465014 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:41.464904    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 14:18:41.465014 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:41.464924    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls podName:ec673038-4e58-4f33-bafe-8155ac3a4efc nodeName:}" failed. No retries permitted until 2026-07-21 14:18:43.464909237 +0000 UTC m=+49.599304588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4plps" (UID: "ec673038-4e58-4f33-bafe-8155ac3a4efc") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:18:41.465014 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:18:41.464966    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm podName:ec673038-4e58-4f33-bafe-8155ac3a4efc nodeName:}" failed. No retries permitted until 2026-07-21 14:18:43.464947529 +0000 UTC m=+49.599342899 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4plps" (UID: "ec673038-4e58-4f33-bafe-8155ac3a4efc") : configmap "kube-rbac-proxy" not found
Jul 21 14:18:41.829228 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:41.829159    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n99kh_0e335d36-8172-432c-965c-7176028cf41f/dns-node-resolver/0.log"
Jul 21 14:18:42.029234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:42.029208    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5888557764-m6s9h_062b0576-1508-4eed-b37b-cb1f0b1ae2b1/registry/0.log"
Jul 21 14:18:42.628884 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:42.628858    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7bvzz_b5a651ff-94a7-45df-a9a9-b0266bf28223/node-ca/0.log"
Jul 21 14:18:43.480299 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.480262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:43.480552 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.480313    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:43.480881 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.480859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec673038-4e58-4f33-bafe-8155ac3a4efc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:43.482716 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.482696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec673038-4e58-4f33-bafe-8155ac3a4efc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4plps\" (UID: \"ec673038-4e58-4f33-bafe-8155ac3a4efc\") " pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:43.617933 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.617904    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4plps"
Jul 21 14:18:43.798068 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.798037    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4plps"]
Jul 21 14:18:43.801970 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:18:43.801946    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec673038_4e58_4f33_bafe_8155ac3a4efc.slice/crio-772744a8b591d0ef5b6ef97101113a1cc322f495ede7ad6b7a77e3f49dceeee2 WatchSource:0}: Error finding container 772744a8b591d0ef5b6ef97101113a1cc322f495ede7ad6b7a77e3f49dceeee2: Status 404 returned error can't find the container with id 772744a8b591d0ef5b6ef97101113a1cc322f495ede7ad6b7a77e3f49dceeee2
Jul 21 14:18:43.829116 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:43.829086    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zm5zn_85ce1fac-29e6-46c0-8aed-8a6f33357deb/serve-healthcheck-canary/0.log"
Jul 21 14:18:44.660851 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:44.660774    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4plps" event={"ID":"ec673038-4e58-4f33-bafe-8155ac3a4efc","Type":"ContainerStarted","Data":"772744a8b591d0ef5b6ef97101113a1cc322f495ede7ad6b7a77e3f49dceeee2"}
Jul 21 14:18:49.652098 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:49.652073    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m4ftr"
Jul 21 14:18:49.675969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:49.675941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4plps" event={"ID":"ec673038-4e58-4f33-bafe-8155ac3a4efc","Type":"ContainerStarted","Data":"5c9786656289cb13344c0c913c654bac8a2a66c3a7f65570b3aa158a05fdfc2e"}
Jul 21 14:18:52.392646 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.392607    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f4bdd79c4-fxqmb"]
Jul 21 14:18:52.395970 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.395941    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.398355 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398169    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 14:18:52.398355 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398173    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 14:18:52.398355 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398192    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 14:18:52.398355 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-br6h4\""
Jul 21 14:18:52.399026 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398855    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 14:18:52.399026 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398899    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 14:18:52.399026 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.398932    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 14:18:52.399209 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.399078    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 14:18:52.404498 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.404481    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 14:18:52.406697 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.406672    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f4bdd79c4-fxqmb"]
Jul 21 14:18:52.449724 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-console-config\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.449724 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-trusted-ca-bundle\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.449976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449751    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-oauth-config\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.449976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-service-ca\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.449976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449851    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-oauth-serving-cert\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.449976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-serving-cert\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.449976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.449919    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snrlf\" (UniqueName: \"kubernetes.io/projected/92e8e65e-7808-492a-84c2-b50b11edabfc-kube-api-access-snrlf\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.550785 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-console-config\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.550785 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550790    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-trusted-ca-bundle\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551031 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550817    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-oauth-config\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551031 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-service-ca\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551031 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-oauth-serving-cert\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551031 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-serving-cert\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551031 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.550920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snrlf\" (UniqueName: \"kubernetes.io/projected/92e8e65e-7808-492a-84c2-b50b11edabfc-kube-api-access-snrlf\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551615 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.551559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-console-config\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551823 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.551785    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-trusted-ca-bundle\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551910 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.551841    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-oauth-serving-cert\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.551962 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.551933    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-service-ca\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.553952 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.553891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-serving-cert\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.554305 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.554287    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-oauth-config\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.561103 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.561081    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snrlf\" (UniqueName: \"kubernetes.io/projected/92e8e65e-7808-492a-84c2-b50b11edabfc-kube-api-access-snrlf\") pod \"console-5f4bdd79c4-fxqmb\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") " pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:52.706830 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:52.706788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:18:53.115463 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:53.115411    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f4bdd79c4-fxqmb"]
Jul 21 14:18:53.119269 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:18:53.119244    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e8e65e_7808_492a_84c2_b50b11edabfc.slice/crio-217d27cec55217263ec9ab388412c4cb4c59373e58f068a04b2bcafb63229871 WatchSource:0}: Error finding container 217d27cec55217263ec9ab388412c4cb4c59373e58f068a04b2bcafb63229871: Status 404 returned error can't find the container with id 217d27cec55217263ec9ab388412c4cb4c59373e58f068a04b2bcafb63229871
Jul 21 14:18:53.690395 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:53.690352    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdd79c4-fxqmb" event={"ID":"92e8e65e-7808-492a-84c2-b50b11edabfc","Type":"ContainerStarted","Data":"217d27cec55217263ec9ab388412c4cb4c59373e58f068a04b2bcafb63229871"}
Jul 21 14:18:53.692573 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:53.692545    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4plps" event={"ID":"ec673038-4e58-4f33-bafe-8155ac3a4efc","Type":"ContainerStarted","Data":"4f83524382a7551baa152b6f80e90f56a0064b0849f05adb84bb1e413ab4a02d"}
Jul 21 14:18:55.611571 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:55.611541    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqzvh"
Jul 21 14:18:56.639199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:56.639174    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5888557764-m6s9h"
Jul 21 14:18:58.711371 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:58.711336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdd79c4-fxqmb" event={"ID":"92e8e65e-7808-492a-84c2-b50b11edabfc","Type":"ContainerStarted","Data":"4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3"}
Jul 21 14:18:58.713475 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:58.713448    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4plps" event={"ID":"ec673038-4e58-4f33-bafe-8155ac3a4efc","Type":"ContainerStarted","Data":"0b30c636f85287ccfb5a860895ab290599493e59143972a9da991e6f75be3379"}
Jul 21 14:18:58.733960 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:58.733914    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f4bdd79c4-fxqmb" podStartSLOduration=2.066360453 podStartE2EDuration="6.73390037s" totalImagesPullingTime="4.667539917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:58.733470457 +0000 UTC m=+64.867865832" watchObservedRunningTime="2026-07-21 14:18:58.73390037 +0000 UTC m=+64.868295744"
Jul 21 14:18:58.756701 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:18:58.756515    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4plps" podStartSLOduration=5.885931334 podStartE2EDuration="19.756500789s" totalImagesPullingTime="13.870569455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:18:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:18:58.756172415 +0000 UTC m=+64.890567788" watchObservedRunningTime="2026-07-21 14:18:58.756500789 +0000 UTC m=+64.890896162"
Jul 21 14:19:00.109504 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.109452    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:19:00.109902 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.109551    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:19:00.111813 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.111796    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 14:19:00.112240 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.112224    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 14:19:00.121982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.121946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e23ecc9-b2ec-4ce2-b581-6b00e8314017-metrics-certs\") pod \"network-metrics-daemon-zjckf\" (UID: \"9e23ecc9-b2ec-4ce2-b581-6b00e8314017\") " pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:19:00.122085 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.121995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55a3aa06-f532-4817-a3ad-dfbcf037ec5a-original-pull-secret\") pod \"global-pull-secret-syncer-24x6m\" (UID: \"55a3aa06-f532-4817-a3ad-dfbcf037ec5a\") " pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:19:00.210595 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.210560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:19:00.213006 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.212987    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 14:19:00.223542 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.223525    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 14:19:00.234654 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.234627    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfdzz\" (UniqueName: \"kubernetes.io/projected/a52903c5-7ce8-423b-aeb9-2ee6079753d6-kube-api-access-dfdzz\") pod \"network-check-target-x296g\" (UID: \"a52903c5-7ce8-423b-aeb9-2ee6079753d6\") " pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:19:00.372081 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.371996    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24x6m"
Jul 21 14:19:00.379014 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.378995    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2kbnq\""
Jul 21 14:19:00.386899 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.386874    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjckf"
Jul 21 14:19:00.480287 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.480260    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qcck5\""
Jul 21 14:19:00.488271 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.488244    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:19:00.506678 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.506649    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24x6m"]
Jul 21 14:19:00.509888 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:00.509855    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a3aa06_f532_4817_a3ad_dfbcf037ec5a.slice/crio-da269399a82d2604181b772a645b36140e55556f34b31e63af41a25bc62550ca WatchSource:0}: Error finding container da269399a82d2604181b772a645b36140e55556f34b31e63af41a25bc62550ca: Status 404 returned error can't find the container with id da269399a82d2604181b772a645b36140e55556f34b31e63af41a25bc62550ca
Jul 21 14:19:00.523733 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.523700    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjckf"]
Jul 21 14:19:00.528283 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:00.528256    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e23ecc9_b2ec_4ce2_b581_6b00e8314017.slice/crio-faffd08456929b32c1ece136becedf6186666c8c3d342f083605a99a54a4e56f WatchSource:0}: Error finding container faffd08456929b32c1ece136becedf6186666c8c3d342f083605a99a54a4e56f: Status 404 returned error can't find the container with id faffd08456929b32c1ece136becedf6186666c8c3d342f083605a99a54a4e56f
Jul 21 14:19:00.618626 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.618599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x296g"]
Jul 21 14:19:00.621667 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:00.621645    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52903c5_7ce8_423b_aeb9_2ee6079753d6.slice/crio-b5565eea13e036831620ba95372ad8541b9dd1930cbc13a9aee8967490c791f4 WatchSource:0}: Error finding container b5565eea13e036831620ba95372ad8541b9dd1930cbc13a9aee8967490c791f4: Status 404 returned error can't find the container with id b5565eea13e036831620ba95372ad8541b9dd1930cbc13a9aee8967490c791f4
Jul 21 14:19:00.720493 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.720450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjckf" event={"ID":"9e23ecc9-b2ec-4ce2-b581-6b00e8314017","Type":"ContainerStarted","Data":"faffd08456929b32c1ece136becedf6186666c8c3d342f083605a99a54a4e56f"}
Jul 21 14:19:00.721568 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.721544    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x296g" event={"ID":"a52903c5-7ce8-423b-aeb9-2ee6079753d6","Type":"ContainerStarted","Data":"b5565eea13e036831620ba95372ad8541b9dd1930cbc13a9aee8967490c791f4"}
Jul 21 14:19:00.722609 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:00.722580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24x6m" event={"ID":"55a3aa06-f532-4817-a3ad-dfbcf037ec5a","Type":"ContainerStarted","Data":"da269399a82d2604181b772a645b36140e55556f34b31e63af41a25bc62550ca"}
Jul 21 14:19:02.708213 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.708174    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:19:02.708629 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.708240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:19:02.712901 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.712877    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:19:02.732009 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.731978    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjckf" event={"ID":"9e23ecc9-b2ec-4ce2-b581-6b00e8314017","Type":"ContainerStarted","Data":"dd4af688b41c4eacbe908c3565fa9ce0f3fc3b1013f676bdb3b2cd39cc690ea4"}
Jul 21 14:19:02.732164 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.732030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjckf" event={"ID":"9e23ecc9-b2ec-4ce2-b581-6b00e8314017","Type":"ContainerStarted","Data":"65803e5d45983354c3888265d4271c70e14b15d37d4c101358b2c0819559dbc7"}
Jul 21 14:19:02.736198 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.736177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:19:02.749812 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:02.749768    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zjckf" podStartSLOduration=66.410494838 podStartE2EDuration="1m7.749753256s" totalImagesPullingTime="1.339258418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:02.748949873 +0000 UTC m=+68.883345241" watchObservedRunningTime="2026-07-21 14:19:02.749753256 +0000 UTC m=+68.884148629"
Jul 21 14:19:06.935159 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:06.935123    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"]
Jul 21 14:19:06.938238 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:06.938214    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:06.942489 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.942449    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-dockercfg-j4cgq\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j4cgq\"" type="*v1.Secret"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944636    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"metrics-client-ca\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944644    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-tls\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" type="*v1.Secret"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944642    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944662    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944643    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-state-metrics-56c5594674-2zxbg\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" podUID="7c00c960-ce49-4140-95ac-fe0d22a01c97" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944687    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 21 14:19:06.944711 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:06.944694    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-state-metrics-custom-resource-state-configmap\" is forbidden: User \"system:node:ip-10-0-129-87.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-87.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" type="*v1.ConfigMap"
Jul 21 14:19:06.960563 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:06.960535    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"]
Jul 21 14:19:06.997006 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:06.996979    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qdd9p"]
Jul 21 14:19:06.999490 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:06.999468    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.002030 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.002004    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 14:19:07.002125 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.002101    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nlq6z\""
Jul 21 14:19:07.002430 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.002402    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 14:19:07.002515 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.002411    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 14:19:07.059438 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059386    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.059611 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-root\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059611 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8407a2ae-c501-42e8-ab18-68e02523212a-metrics-client-ca\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059611 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059589    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059738 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzfhs\" (UniqueName: \"kubernetes.io/projected/8407a2ae-c501-42e8-ab18-68e02523212a-kube-api-access-xzfhs\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059738 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059691    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-textfile\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059738 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c00c960-ce49-4140-95ac-fe0d22a01c97-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.059858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-wtmp\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-sys\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.059858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5nw8\" (UniqueName: \"kubernetes.io/projected/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-api-access-h5nw8\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.059858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.060066 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059859    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.060066 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-tls\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.060066 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.059957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7c00c960-ce49-4140-95ac-fe0d22a01c97-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-tls\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161251    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7c00c960-ce49-4140-95ac-fe0d22a01c97-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161321    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161342    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-root\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8407a2ae-c501-42e8-ab18-68e02523212a-metrics-client-ca\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161388 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:07.161375    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-root\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:07.161507    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-tls podName:8407a2ae-c501-42e8-ab18-68e02523212a nodeName:}" failed. No retries permitted until 2026-07-21 14:19:07.661473547 +0000 UTC m=+73.795868900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-tls") pod "node-exporter-qdd9p" (UID: "8407a2ae-c501-42e8-ab18-68e02523212a") : secret "node-exporter-tls" not found
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzfhs\" (UniqueName: \"kubernetes.io/projected/8407a2ae-c501-42e8-ab18-68e02523212a-kube-api-access-xzfhs\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-textfile\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c00c960-ce49-4140-95ac-fe0d22a01c97-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-wtmp\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161680    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-sys\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7c00c960-ce49-4140-95ac-fe0d22a01c97-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.161771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5nw8\" (UniqueName: \"kubernetes.io/projected/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-api-access-h5nw8\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:07.162197 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161801    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.162197 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161907    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-wtmp\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.162197 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8407a2ae-c501-42e8-ab18-68e02523212a-sys\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.162197 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.161994    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-textfile\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.164309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.164281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.665144 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.665109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-tls\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.668192 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.668160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8407a2ae-c501-42e8-ab18-68e02523212a-node-exporter-tls\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.883676 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.883656    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 14:19:07.892887 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.892866    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8407a2ae-c501-42e8-ab18-68e02523212a-metrics-client-ca\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:07.892982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:07.892959    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c00c960-ce49-4140-95ac-fe0d22a01c97-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.090240 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.090208    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 14:19:08.094500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.093944    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.099948 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.099923    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 21 14:19:08.100132 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.100116    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 14:19:08.100206 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.100153    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 14:19:08.100206 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.100187    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 14:19:08.100330 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.100209    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 14:19:08.100483 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.100466    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 14:19:08.100910 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.100866    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 14:19:08.101479 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.101461    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p755w\""
Jul 21 14:19:08.101573 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.101557    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 14:19:08.101682 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.101667    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 14:19:08.101765 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.101748    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 14:19:08.101877 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.101860    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.106620 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.106607    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 14:19:08.110867 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.110846    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 14:19:08.125478 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.125460    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 21 14:19:08.134306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.134282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.154555 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.154533    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j4cgq\""
Jul 21 14:19:08.161922 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.161904    2569 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-kube-rbac-proxy-config: failed to sync secret cache: timed out waiting for the condition
Jul 21 14:19:08.162003 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.161979    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-kube-rbac-proxy-config podName:7c00c960-ce49-4140-95ac-fe0d22a01c97 nodeName:}" failed. No retries permitted until 2026-07-21 14:19:08.661961263 +0000 UTC m=+74.796356614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-kube-rbac-proxy-config" (UniqueName: "kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-kube-rbac-proxy-config") pod "kube-state-metrics-56c5594674-2zxbg" (UID: "7c00c960-ce49-4140-95ac-fe0d22a01c97") : failed to sync secret cache: timed out waiting for the condition
Jul 21 14:19:08.169521 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169576 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169576 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169576 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169666 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169698 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169658    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169737 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169737 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169725    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsftg\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-kube-api-access-gsftg\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169796 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169748    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-config-out\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169796 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-web-config\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169890 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.169890 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.169873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.177606 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.177590    2569 projected.go:291] Couldn't get configMap openshift-monitoring/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 21 14:19:08.177651 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.177615    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xzfhs for pod openshift-monitoring/node-exporter-qdd9p: failed to sync configmap cache: timed out waiting for the condition
Jul 21 14:19:08.177688 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.177662    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8407a2ae-c501-42e8-ab18-68e02523212a-kube-api-access-xzfhs podName:8407a2ae-c501-42e8-ab18-68e02523212a nodeName:}" failed. No retries permitted until 2026-07-21 14:19:08.677650145 +0000 UTC m=+74.812045496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xzfhs" (UniqueName: "kubernetes.io/projected/8407a2ae-c501-42e8-ab18-68e02523212a-kube-api-access-xzfhs") pod "node-exporter-qdd9p" (UID: "8407a2ae-c501-42e8-ab18-68e02523212a") : failed to sync configmap cache: timed out waiting for the condition
Jul 21 14:19:08.180388 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.180376    2569 projected.go:291] Couldn't get configMap openshift-monitoring/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 21 14:19:08.180439 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.180397    2569 projected.go:196] Error preparing data for projected volume kube-api-access-h5nw8 for pod openshift-monitoring/kube-state-metrics-56c5594674-2zxbg: failed to sync configmap cache: timed out waiting for the condition
Jul 21 14:19:08.180474 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:08.180451    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-api-access-h5nw8 podName:7c00c960-ce49-4140-95ac-fe0d22a01c97 nodeName:}" failed. No retries permitted until 2026-07-21 14:19:08.680440129 +0000 UTC m=+74.814835481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h5nw8" (UniqueName: "kubernetes.io/projected/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-api-access-h5nw8") pod "kube-state-metrics-56c5594674-2zxbg" (UID: "7c00c960-ce49-4140-95ac-fe0d22a01c97") : failed to sync configmap cache: timed out waiting for the condition
Jul 21 14:19:08.270916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.270894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.270923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsftg\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-kube-api-access-gsftg\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.270944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-config-out\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.270966    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-web-config\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.270990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271020    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271267 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271046    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271267 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271267 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271267 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.271267 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.272088 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.271918    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.272273 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.272249    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.272390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.272369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.272555 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.272534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.273190 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.273160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.274349 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.274303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-config-out\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.274584 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.274563    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.274787 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.274760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.275049 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.275005    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-web-config\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.275049 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.275027    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.275319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.275298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.275560 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.275544    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.275726 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.275712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.276385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.276365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.350111 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.350039    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 21 14:19:08.450521 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.450495    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 14:19:08.460693 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.460668    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsftg\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-kube-api-access-gsftg\") pod \"alertmanager-main-0\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.676565 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.676473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.679024 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.678999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.703486 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.703462    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:19:08.756863 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.756674    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x296g" event={"ID":"a52903c5-7ce8-423b-aeb9-2ee6079753d6","Type":"ContainerStarted","Data":"bcd4b4d9ac76fa8ea97aade6564f92965a58da7ef3cdd255a492176a9177dd93"}
Jul 21 14:19:08.756863 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.756785    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:19:08.758332 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.758309    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24x6m" event={"ID":"55a3aa06-f532-4817-a3ad-dfbcf037ec5a","Type":"ContainerStarted","Data":"70c3268251afe1f46fc7d488d7b1af2ac67cc92eb840b08ce88b3b8466606cdb"}
Jul 21 14:19:08.775109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.774379    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x296g" podStartSLOduration=66.568734304 podStartE2EDuration="1m13.774362688s" totalImagesPullingTime="7.205628384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:08.774268059 +0000 UTC m=+74.908663460" watchObservedRunningTime="2026-07-21 14:19:08.774362688 +0000 UTC m=+74.908758062"
Jul 21 14:19:08.777372 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.777348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5nw8\" (UniqueName: \"kubernetes.io/projected/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-api-access-h5nw8\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.777514 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.777402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzfhs\" (UniqueName: \"kubernetes.io/projected/8407a2ae-c501-42e8-ab18-68e02523212a-kube-api-access-xzfhs\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:08.779923 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.779901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5nw8\" (UniqueName: \"kubernetes.io/projected/7c00c960-ce49-4140-95ac-fe0d22a01c97-kube-api-access-h5nw8\") pod \"kube-state-metrics-56c5594674-2zxbg\" (UID: \"7c00c960-ce49-4140-95ac-fe0d22a01c97\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:08.780037 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.779905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzfhs\" (UniqueName: \"kubernetes.io/projected/8407a2ae-c501-42e8-ab18-68e02523212a-kube-api-access-xzfhs\") pod \"node-exporter-qdd9p\" (UID: \"8407a2ae-c501-42e8-ab18-68e02523212a\") " pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:08.793971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.793925    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-24x6m" podStartSLOduration=66.473396192 podStartE2EDuration="1m13.793909711s" totalImagesPullingTime="7.320513519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:08.7929724 +0000 UTC m=+74.927367774" watchObservedRunningTime="2026-07-21 14:19:08.793909711 +0000 UTC m=+74.928305092"
Jul 21 14:19:08.811151 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.811125    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qdd9p"
Jul 21 14:19:08.821162 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:08.821138    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8407a2ae_c501_42e8_ab18_68e02523212a.slice/crio-92dfea6e738228dfeee7fc9285f042d11efc8eb53cb588ea1728464466e531b6 WatchSource:0}: Error finding container 92dfea6e738228dfeee7fc9285f042d11efc8eb53cb588ea1728464466e531b6: Status 404 returned error can't find the container with id 92dfea6e738228dfeee7fc9285f042d11efc8eb53cb588ea1728464466e531b6
Jul 21 14:19:08.851671 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:08.851640    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 14:19:08.855358 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:08.855334    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04e637f_06e2_415e_8feb_94f8534faa0a.slice/crio-6d49d792b8aa11bb96f791ed0427236fbf50ab454fc0dd0b44365bc27d80d1ed WatchSource:0}: Error finding container 6d49d792b8aa11bb96f791ed0427236fbf50ab454fc0dd0b44365bc27d80d1ed: Status 404 returned error can't find the container with id 6d49d792b8aa11bb96f791ed0427236fbf50ab454fc0dd0b44365bc27d80d1ed
Jul 21 14:19:09.048428 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:09.048372    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"
Jul 21 14:19:09.175160 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:09.175125    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-2zxbg"]
Jul 21 14:19:09.178335 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:09.178300    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c00c960_ce49_4140_95ac_fe0d22a01c97.slice/crio-1a6f2ecce4fa99eb8c5b2a976f704b7637275f80f47b60cae6929e12ec790c8a WatchSource:0}: Error finding container 1a6f2ecce4fa99eb8c5b2a976f704b7637275f80f47b60cae6929e12ec790c8a: Status 404 returned error can't find the container with id 1a6f2ecce4fa99eb8c5b2a976f704b7637275f80f47b60cae6929e12ec790c8a
Jul 21 14:19:09.764323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:09.764284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg" event={"ID":"7c00c960-ce49-4140-95ac-fe0d22a01c97","Type":"ContainerStarted","Data":"1a6f2ecce4fa99eb8c5b2a976f704b7637275f80f47b60cae6929e12ec790c8a"}
Jul 21 14:19:09.765707 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:09.765671    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdd9p" event={"ID":"8407a2ae-c501-42e8-ab18-68e02523212a","Type":"ContainerStarted","Data":"92dfea6e738228dfeee7fc9285f042d11efc8eb53cb588ea1728464466e531b6"}
Jul 21 14:19:09.767562 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:09.767536    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"6d49d792b8aa11bb96f791ed0427236fbf50ab454fc0dd0b44365bc27d80d1ed"}
Jul 21 14:19:10.773380 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:10.773342    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdd9p" event={"ID":"8407a2ae-c501-42e8-ab18-68e02523212a","Type":"ContainerStarted","Data":"de0af53efe5dfaef30c03875dbe9772ee71c2abfbd8146d7c5c64a6396984b9b"}
Jul 21 14:19:10.775705 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:10.775549    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682" exitCode=0
Jul 21 14:19:10.775705 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:10.775614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"}
Jul 21 14:19:11.349101 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.349070    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"]
Jul 21 14:19:11.351544 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.351513    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.353613 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.353589    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3j2nja2d86f2o\""
Jul 21 14:19:11.353613 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.353609    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 21 14:19:11.353809 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.353696    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 21 14:19:11.353809 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.353779    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 21 14:19:11.353809 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.353781    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7jjdg\""
Jul 21 14:19:11.354200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.354182    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 21 14:19:11.362193 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.362155    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"]
Jul 21 14:19:11.401821 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.401790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/15449fe8-6595-4138-8a44-4e79336a42a9-audit-log\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.401995 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.401832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-client-ca-bundle\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.401995 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.401857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-secret-metrics-server-tls\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.401995 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.401908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpczx\" (UniqueName: \"kubernetes.io/projected/15449fe8-6595-4138-8a44-4e79336a42a9-kube-api-access-jpczx\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.401995 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.401980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-secret-metrics-server-client-certs\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.402196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.402012    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/15449fe8-6595-4138-8a44-4e79336a42a9-metrics-server-audit-profiles\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.402196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.402082    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15449fe8-6595-4138-8a44-4e79336a42a9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15449fe8-6595-4138-8a44-4e79336a42a9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503257 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503154    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/15449fe8-6595-4138-8a44-4e79336a42a9-audit-log\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503257 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-client-ca-bundle\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503257 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503219    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-secret-metrics-server-tls\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503432 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpczx\" (UniqueName: \"kubernetes.io/projected/15449fe8-6595-4138-8a44-4e79336a42a9-kube-api-access-jpczx\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503487    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-secret-metrics-server-client-certs\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.503570 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.503515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/15449fe8-6595-4138-8a44-4e79336a42a9-metrics-server-audit-profiles\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.504519 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.504277    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15449fe8-6595-4138-8a44-4e79336a42a9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.504519 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.504472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/15449fe8-6595-4138-8a44-4e79336a42a9-metrics-server-audit-profiles\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.504519 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.504480    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/15449fe8-6595-4138-8a44-4e79336a42a9-audit-log\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.506138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.506118    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-secret-metrics-server-tls\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.506249 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.506155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-client-ca-bundle\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.506555 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.506533    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/15449fe8-6595-4138-8a44-4e79336a42a9-secret-metrics-server-client-certs\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.516061 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.516040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpczx\" (UniqueName: \"kubernetes.io/projected/15449fe8-6595-4138-8a44-4e79336a42a9-kube-api-access-jpczx\") pod \"metrics-server-5d86db7f6f-dwrc2\" (UID: \"15449fe8-6595-4138-8a44-4e79336a42a9\") " pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.663007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.662982    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:11.782885 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.782854    2569 generic.go:356] "Generic (PLEG): container finished" podID="8407a2ae-c501-42e8-ab18-68e02523212a" containerID="de0af53efe5dfaef30c03875dbe9772ee71c2abfbd8146d7c5c64a6396984b9b" exitCode=0
Jul 21 14:19:11.783309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.782973    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdd9p" event={"ID":"8407a2ae-c501-42e8-ab18-68e02523212a","Type":"ContainerDied","Data":"de0af53efe5dfaef30c03875dbe9772ee71c2abfbd8146d7c5c64a6396984b9b"}
Jul 21 14:19:11.790931 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.790806    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg" event={"ID":"7c00c960-ce49-4140-95ac-fe0d22a01c97","Type":"ContainerStarted","Data":"5c7b1b17bb986db0a16977784098baa50307754736235d427831d65f4dad839a"}
Jul 21 14:19:11.790931 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.790866    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg" event={"ID":"7c00c960-ce49-4140-95ac-fe0d22a01c97","Type":"ContainerStarted","Data":"b10c325c3321edaaec16e41840ff5d3bd47e9d79db0b4bec40d64762491b0da0"}
Jul 21 14:19:11.843231 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:11.841589    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"]
Jul 21 14:19:11.848835 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:11.848811    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15449fe8_6595_4138_8a44_4e79336a42a9.slice/crio-ace6f1712e43bdd6b6bc40961b4433ed8d7cbd2faa56e8cbd71fb4105a8da8d6 WatchSource:0}: Error finding container ace6f1712e43bdd6b6bc40961b4433ed8d7cbd2faa56e8cbd71fb4105a8da8d6: Status 404 returned error can't find the container with id ace6f1712e43bdd6b6bc40961b4433ed8d7cbd2faa56e8cbd71fb4105a8da8d6
Jul 21 14:19:12.797517 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:12.797475    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg" event={"ID":"7c00c960-ce49-4140-95ac-fe0d22a01c97","Type":"ContainerStarted","Data":"0ba8827b192ac2d2996069fe19dd8a5e46fda145d2ff73925105bd3fafd2eb89"}
Jul 21 14:19:12.800644 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:12.800614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdd9p" event={"ID":"8407a2ae-c501-42e8-ab18-68e02523212a","Type":"ContainerStarted","Data":"70102c14eec478bfe348f125f5727bfeb7faa8ad6c32437ea8260d322c579c3b"}
Jul 21 14:19:12.800783 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:12.800672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdd9p" event={"ID":"8407a2ae-c501-42e8-ab18-68e02523212a","Type":"ContainerStarted","Data":"97d264802c8b6a22a54e2aec90f109caf7fd0b0713b7eb9e1d421e6ff3890199"}
Jul 21 14:19:12.801941 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:12.801913    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2" event={"ID":"15449fe8-6595-4138-8a44-4e79336a42a9","Type":"ContainerStarted","Data":"ace6f1712e43bdd6b6bc40961b4433ed8d7cbd2faa56e8cbd71fb4105a8da8d6"}
Jul 21 14:19:12.818767 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:12.818718    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-2zxbg" podStartSLOduration=4.40414568 podStartE2EDuration="6.818698895s" totalImagesPullingTime="2.414553215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:12.817231507 +0000 UTC m=+78.951626883" watchObservedRunningTime="2026-07-21 14:19:12.818698895 +0000 UTC m=+78.953094272"
Jul 21 14:19:12.836735 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:12.836679    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qdd9p" podStartSLOduration=4.976356776 podStartE2EDuration="6.8366647s" totalImagesPullingTime="1.779411432s" totalInitContainerRuntime="80.896492ms" isStatefulPod=true podCreationTimestamp="2026-07-21 14:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:12.835793546 +0000 UTC m=+78.970188947" watchObservedRunningTime="2026-07-21 14:19:12.8366647 +0000 UTC m=+78.971060075"
Jul 21 14:19:13.237178 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.237141    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:19:13.241022 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.240996    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.244516 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.244470    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 21 14:19:13.244662 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.244646    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 21 14:19:13.244948 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.244911    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 21 14:19:13.245053 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.244997    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 21 14:19:13.245053 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245046    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 21 14:19:13.245391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245269    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 21 14:19:13.245391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245294    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 21 14:19:13.245391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245299    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 21 14:19:13.245821 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245671    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gqskc\""
Jul 21 14:19:13.245821 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245676    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f0dp7t9jbjrbj\""
Jul 21 14:19:13.245982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245901    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 21 14:19:13.245982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.245937    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 21 14:19:13.246216 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.246199    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 21 14:19:13.246336 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.246314    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 21 14:19:13.246488 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.246201    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 21 14:19:13.250223 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.250205    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 21 14:19:13.265377 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.265351    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:19:13.319101 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319101 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319126    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-web-config\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319369    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319404 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319404    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319445    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319480    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-config\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftf7x\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-kube-api-access-ftf7x\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319572    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319621    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319641    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-config-out\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.319792 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.319674    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422297 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422209    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.421165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422465    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-config-out\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422563    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.422790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422778    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-web-config\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422890    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.422967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.423000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-config\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.423031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.423091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.423053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftf7x\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-kube-api-access-ftf7x\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.425542 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.423956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.428411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.428633    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.428720    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.428951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.429040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-config\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.429235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.429785 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.429478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.430011 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.429960    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.430915 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.430785    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.431545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.431064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.431545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.431280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.431545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.431505    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.432271 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.432244    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.432360 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.432274    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.432518 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.432492    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.432877 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.432858    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-config-out\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.433342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.433314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-web-config\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.438491 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.438472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftf7x\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-kube-api-access-ftf7x\") pod \"prometheus-k8s-0\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.553198 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.553175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:13.741328 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.741222    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:19:13.744742 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:19:13.744716    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05b263b_f90b_4763_ae00_917f34a14226.slice/crio-7f2d014cd23f2012aa64d70ad6c65023299b835b24df6a9b251a83912a729e1b WatchSource:0}: Error finding container 7f2d014cd23f2012aa64d70ad6c65023299b835b24df6a9b251a83912a729e1b: Status 404 returned error can't find the container with id 7f2d014cd23f2012aa64d70ad6c65023299b835b24df6a9b251a83912a729e1b
Jul 21 14:19:13.809745 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.809712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"}
Jul 21 14:19:13.810129 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.809797    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"}
Jul 21 14:19:13.810129 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.809838    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"}
Jul 21 14:19:13.810129 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.809878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"}
Jul 21 14:19:13.812574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.811840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2" event={"ID":"15449fe8-6595-4138-8a44-4e79336a42a9","Type":"ContainerStarted","Data":"b295dd6841ef1c27c9c3b0854109d6be2e7b0ad7674ad1ec69d2c8a60e0391c8"}
Jul 21 14:19:13.814948 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.814910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"7f2d014cd23f2012aa64d70ad6c65023299b835b24df6a9b251a83912a729e1b"}
Jul 21 14:19:13.841303 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:13.841256    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2" podStartSLOduration=1.093375136 podStartE2EDuration="2.841238752s" totalImagesPullingTime="1.747863616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:13.839964506 +0000 UTC m=+79.974359905" watchObservedRunningTime="2026-07-21 14:19:13.841238752 +0000 UTC m=+79.975634126"
Jul 21 14:19:14.821586 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:14.821454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"}
Jul 21 14:19:14.822969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:14.822933    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8" exitCode=0
Jul 21 14:19:14.823098 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:14.823023    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"}
Jul 21 14:19:15.829824 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:15.829788    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerStarted","Data":"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"}
Jul 21 14:19:15.860168 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:15.860108    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.452079103 podStartE2EDuration="7.860089245s" totalImagesPullingTime="5.392814215s" totalInitContainerRuntime="15.195927ms" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:15.858618862 +0000 UTC m=+81.993014255" watchObservedRunningTime="2026-07-21 14:19:15.860089245 +0000 UTC m=+81.994484996"
Jul 21 14:19:19.850610 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:19.850571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"}
Jul 21 14:19:19.851077 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:19.850656    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"}
Jul 21 14:19:21.865897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:21.862189    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"}
Jul 21 14:19:21.865897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:21.862344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"}
Jul 21 14:19:21.865897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:21.862401    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"}
Jul 21 14:19:21.865897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:21.862492    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerStarted","Data":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"}
Jul 21 14:19:21.898487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:21.898382    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.718247728 podStartE2EDuration="8.8983604s" totalImagesPullingTime="6.156892382s" totalInitContainerRuntime="23.22029ms" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:21.896370556 +0000 UTC m=+88.030765930" watchObservedRunningTime="2026-07-21 14:19:21.8983604 +0000 UTC m=+88.032755774"
Jul 21 14:19:22.170974 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:22.170892    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f4bdd79c4-fxqmb"]
Jul 21 14:19:23.554307 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:23.554277    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:19:31.663953 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:31.663913    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:31.664453 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:31.663986    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:39.770756 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:39.770729    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x296g"
Jul 21 14:19:47.191665 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.191617    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f4bdd79c4-fxqmb" podUID="92e8e65e-7808-492a-84c2-b50b11edabfc" containerName="console" containerID="cri-o://4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3" gracePeriod=15
Jul 21 14:19:47.429673 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.429651    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:19:47.520319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520220    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-trusted-ca-bundle\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520282    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-serving-cert\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520310    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-console-config\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520625 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520354    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-service-ca\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520625 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520463    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-oauth-config\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520625 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520488    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-oauth-serving-cert\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520625 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520530    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snrlf\" (UniqueName: \"kubernetes.io/projected/92e8e65e-7808-492a-84c2-b50b11edabfc-kube-api-access-snrlf\") pod \"92e8e65e-7808-492a-84c2-b50b11edabfc\" (UID: \"92e8e65e-7808-492a-84c2-b50b11edabfc\") "
Jul 21 14:19:47.520815 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520650    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:19:47.520815 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-console-config" (OuterVolumeSpecName: "console-config") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:19:47.520815 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520777    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-trusted-ca-bundle\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.520962 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.520824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-service-ca" (OuterVolumeSpecName: "service-ca") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:19:47.521148 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.521122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:19:47.523042 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.523011    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e8e65e-7808-492a-84c2-b50b11edabfc-kube-api-access-snrlf" (OuterVolumeSpecName: "kube-api-access-snrlf") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "kube-api-access-snrlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:19:47.523122 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.523026    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:19:47.523122 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.523104    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "92e8e65e-7808-492a-84c2-b50b11edabfc" (UID: "92e8e65e-7808-492a-84c2-b50b11edabfc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:19:47.621188 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.621148    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.621188 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.621179    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-oauth-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.621188 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.621189    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-oauth-serving-cert\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.621188 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.621197    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snrlf\" (UniqueName: \"kubernetes.io/projected/92e8e65e-7808-492a-84c2-b50b11edabfc-kube-api-access-snrlf\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.621475 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.621206    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92e8e65e-7808-492a-84c2-b50b11edabfc-console-serving-cert\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.621475 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.621214    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92e8e65e-7808-492a-84c2-b50b11edabfc-console-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:19:47.955461 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.955403    2569 generic.go:356] "Generic (PLEG): container finished" podID="92e8e65e-7808-492a-84c2-b50b11edabfc" containerID="4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3" exitCode=0
Jul 21 14:19:47.955461 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.955459    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdd79c4-fxqmb"
Jul 21 14:19:47.955664 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.955491    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdd79c4-fxqmb" event={"ID":"92e8e65e-7808-492a-84c2-b50b11edabfc","Type":"ContainerDied","Data":"4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3"}
Jul 21 14:19:47.955664 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.955541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdd79c4-fxqmb" event={"ID":"92e8e65e-7808-492a-84c2-b50b11edabfc","Type":"ContainerDied","Data":"217d27cec55217263ec9ab388412c4cb4c59373e58f068a04b2bcafb63229871"}
Jul 21 14:19:47.955664 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.955572    2569 scope.go:117] "RemoveContainer" containerID="4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3"
Jul 21 14:19:47.965091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.965073    2569 scope.go:117] "RemoveContainer" containerID="4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3"
Jul 21 14:19:47.965397 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:19:47.965377    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3\": container with ID starting with 4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3 not found: ID does not exist" containerID="4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3"
Jul 21 14:19:47.965475 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.965406    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3"} err="failed to get container status \"4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3\": rpc error: code = NotFound desc = could not find container \"4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3\": container with ID starting with 4bd41fb045ca5d50bb43228270f7f7edb1443181208209895c24f1e02064aff3 not found: ID does not exist"
Jul 21 14:19:47.979101 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.979077    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f4bdd79c4-fxqmb"]
Jul 21 14:19:47.983870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:47.983848    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f4bdd79c4-fxqmb"]
Jul 21 14:19:48.466840 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:48.466809    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e8e65e-7808-492a-84c2-b50b11edabfc" path="/var/lib/kubelet/pods/92e8e65e-7808-492a-84c2-b50b11edabfc/volumes"
Jul 21 14:19:51.669233 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:51.669208    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:19:51.673010 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:19:51.672987    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d86db7f6f-dwrc2"
Jul 21 14:20:13.553657 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:13.553615    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:13.574267 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:13.574235    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:14.062879 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:14.062851    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:27.320554 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320513    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 14:20:27.321071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320837    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="alertmanager" containerID="cri-o://b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c" gracePeriod=120
Jul 21 14:20:27.321071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320921    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-metric" containerID="cri-o://cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71" gracePeriod=120
Jul 21 14:20:27.321071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320953    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-web" containerID="cri-o://45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678" gracePeriod=120
Jul 21 14:20:27.321071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320981    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="prom-label-proxy" containerID="cri-o://d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3" gracePeriod=120
Jul 21 14:20:27.321071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320996    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy" containerID="cri-o://b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0" gracePeriod=120
Jul 21 14:20:27.321071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:27.320990    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="config-reloader" containerID="cri-o://28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed" gracePeriod=120
Jul 21 14:20:28.097658 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097620    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3" exitCode=0
Jul 21 14:20:28.097658 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097653    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0" exitCode=0
Jul 21 14:20:28.097658 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097660    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed" exitCode=0
Jul 21 14:20:28.097658 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097666    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c" exitCode=0
Jul 21 14:20:28.097922 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097686    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"}
Jul 21 14:20:28.097922 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097745    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"}
Jul 21 14:20:28.097922 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097772    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"}
Jul 21 14:20:28.097922 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.097797    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"}
Jul 21 14:20:28.552297 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.552273    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:20:28.558768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558746    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-config-volume\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.558887 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558781    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.558887 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558826    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-trusted-ca-bundle\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.558887 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558858    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.558887 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558881    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-main-tls\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559108 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558911    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-web-config\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559108 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558940    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559108 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.558968    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-cluster-tls-config\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559108 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559003    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-metrics-client-ca\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559108 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559052    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-config-out\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559108 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559084    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsftg\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-kube-api-access-gsftg\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559124    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-main-db\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559148    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-tls-assets\") pod \"c04e637f-06e2-415e-8feb-94f8534faa0a\" (UID: \"c04e637f-06e2-415e-8feb-94f8534faa0a\") "
Jul 21 14:20:28.559381 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559249    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:28.559609 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.559588    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.560606 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.560545    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:20:28.560606 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.560553    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:28.562359 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.562332    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:20:28.562829 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.562664    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.562829 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.562774    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-config-volume" (OuterVolumeSpecName: "config-volume") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.562829 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.562793    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-config-out" (OuterVolumeSpecName: "config-out") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:20:28.564022 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.563980    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-kube-api-access-gsftg" (OuterVolumeSpecName: "kube-api-access-gsftg") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "kube-api-access-gsftg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:20:28.564126 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.564019    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.564432 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.564380    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.564564 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.564535    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.567983 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.567933    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.574000 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.573979    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-web-config" (OuterVolumeSpecName: "web-config") pod "c04e637f-06e2-415e-8feb-94f8534faa0a" (UID: "c04e637f-06e2-415e-8feb-94f8534faa0a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660408    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-cluster-tls-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660457    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04e637f-06e2-415e-8feb-94f8534faa0a-metrics-client-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660468    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-config-out\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660477    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsftg\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-kube-api-access-gsftg\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660487    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04e637f-06e2-415e-8feb-94f8534faa0a-alertmanager-main-db\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660495    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04e637f-06e2-415e-8feb-94f8534faa0a-tls-assets\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660502    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-config-volume\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660811 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660512    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660811 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660521    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660811 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660531    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-main-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660811 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660541    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-web-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:28.660811 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:28.660549    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04e637f-06e2-415e-8feb-94f8534faa0a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:29.103446 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103392    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71" exitCode=0
Jul 21 14:20:29.103446 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103438    2569 generic.go:356] "Generic (PLEG): container finished" podID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerID="45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678" exitCode=0
Jul 21 14:20:29.103684 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103477    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 14:20:29.103684 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"}
Jul 21 14:20:29.103684 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"}
Jul 21 14:20:29.103684 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103613    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04e637f-06e2-415e-8feb-94f8534faa0a","Type":"ContainerDied","Data":"6d49d792b8aa11bb96f791ed0427236fbf50ab454fc0dd0b44365bc27d80d1ed"}
Jul 21 14:20:29.103684 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.103632    2569 scope.go:117] "RemoveContainer" containerID="d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"
Jul 21 14:20:29.110910 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.110882    2569 scope.go:117] "RemoveContainer" containerID="cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"
Jul 21 14:20:29.119911 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.119893    2569 scope.go:117] "RemoveContainer" containerID="b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"
Jul 21 14:20:29.127984 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.127738    2569 scope.go:117] "RemoveContainer" containerID="45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"
Jul 21 14:20:29.134732 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.134713    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 14:20:29.135874 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.135854    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 14:20:29.136827 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.136808    2569 scope.go:117] "RemoveContainer" containerID="28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"
Jul 21 14:20:29.143729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.143714    2569 scope.go:117] "RemoveContainer" containerID="b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"
Jul 21 14:20:29.150534 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.150516    2569 scope.go:117] "RemoveContainer" containerID="a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"
Jul 21 14:20:29.157359 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.157330    2569 scope.go:117] "RemoveContainer" containerID="d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"
Jul 21 14:20:29.157611 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.157588    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3\": container with ID starting with d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3 not found: ID does not exist" containerID="d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"
Jul 21 14:20:29.157664 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.157621    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"} err="failed to get container status \"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3\": rpc error: code = NotFound desc = could not find container \"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3\": container with ID starting with d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3 not found: ID does not exist"
Jul 21 14:20:29.157664 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.157640    2569 scope.go:117] "RemoveContainer" containerID="cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"
Jul 21 14:20:29.157882 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.157864    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71\": container with ID starting with cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71 not found: ID does not exist" containerID="cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"
Jul 21 14:20:29.157928 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.157890    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"} err="failed to get container status \"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71\": rpc error: code = NotFound desc = could not find container \"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71\": container with ID starting with cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71 not found: ID does not exist"
Jul 21 14:20:29.157928 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.157906    2569 scope.go:117] "RemoveContainer" containerID="b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"
Jul 21 14:20:29.158107 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.158091    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0\": container with ID starting with b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0 not found: ID does not exist" containerID="b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"
Jul 21 14:20:29.158149 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158112    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"} err="failed to get container status \"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0\": rpc error: code = NotFound desc = could not find container \"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0\": container with ID starting with b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0 not found: ID does not exist"
Jul 21 14:20:29.158149 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158126    2569 scope.go:117] "RemoveContainer" containerID="45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"
Jul 21 14:20:29.158344 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.158330    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678\": container with ID starting with 45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678 not found: ID does not exist" containerID="45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"
Jul 21 14:20:29.158394 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158345    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"} err="failed to get container status \"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678\": rpc error: code = NotFound desc = could not find container \"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678\": container with ID starting with 45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678 not found: ID does not exist"
Jul 21 14:20:29.158394 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158357    2569 scope.go:117] "RemoveContainer" containerID="28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"
Jul 21 14:20:29.158584 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.158568    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed\": container with ID starting with 28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed not found: ID does not exist" containerID="28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"
Jul 21 14:20:29.158622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158588    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"} err="failed to get container status \"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed\": rpc error: code = NotFound desc = could not find container \"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed\": container with ID starting with 28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed not found: ID does not exist"
Jul 21 14:20:29.158622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158602    2569 scope.go:117] "RemoveContainer" containerID="b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"
Jul 21 14:20:29.158817 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.158801    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c\": container with ID starting with b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c not found: ID does not exist" containerID="b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"
Jul 21 14:20:29.158853 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158820    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"} err="failed to get container status \"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c\": rpc error: code = NotFound desc = could not find container \"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c\": container with ID starting with b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c not found: ID does not exist"
Jul 21 14:20:29.158853 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.158832    2569 scope.go:117] "RemoveContainer" containerID="a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"
Jul 21 14:20:29.159080 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:29.159052    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682\": container with ID starting with a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682 not found: ID does not exist" containerID="a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"
Jul 21 14:20:29.159212 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159086    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"} err="failed to get container status \"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682\": rpc error: code = NotFound desc = could not find container \"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682\": container with ID starting with a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682 not found: ID does not exist"
Jul 21 14:20:29.159212 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159111    2569 scope.go:117] "RemoveContainer" containerID="d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"
Jul 21 14:20:29.159387 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159364    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3"} err="failed to get container status \"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3\": rpc error: code = NotFound desc = could not find container \"d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3\": container with ID starting with d3181c92839ca59f28bd0c296334580ecd30994e5f0c6816415285a6dbf395e3 not found: ID does not exist"
Jul 21 14:20:29.159496 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159390    2569 scope.go:117] "RemoveContainer" containerID="cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"
Jul 21 14:20:29.159749 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159724    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71"} err="failed to get container status \"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71\": rpc error: code = NotFound desc = could not find container \"cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71\": container with ID starting with cffc39c45e97cf45af6dc90b070942c78f0069b7bb6cb182141d368d57928a71 not found: ID does not exist"
Jul 21 14:20:29.159831 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159751    2569 scope.go:117] "RemoveContainer" containerID="b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"
Jul 21 14:20:29.160015 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.159992    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0"} err="failed to get container status \"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0\": rpc error: code = NotFound desc = could not find container \"b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0\": container with ID starting with b08bdc968a03f9d52cb10da6b303cd94792ad5ba4598d10338d07d9a4a3250e0 not found: ID does not exist"
Jul 21 14:20:29.160015 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160016    2569 scope.go:117] "RemoveContainer" containerID="45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"
Jul 21 14:20:29.160300 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160279    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678"} err="failed to get container status \"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678\": rpc error: code = NotFound desc = could not find container \"45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678\": container with ID starting with 45fb583bc5761518eaf6087d91ba296e35ee24c5fe5ecec83817b408c3d57678 not found: ID does not exist"
Jul 21 14:20:29.160350 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160303    2569 scope.go:117] "RemoveContainer" containerID="28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"
Jul 21 14:20:29.160539 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160522    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed"} err="failed to get container status \"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed\": rpc error: code = NotFound desc = could not find container \"28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed\": container with ID starting with 28bf73494637e9a0abab9ed866367a69c4aab1c41e26228bc3d129f702be2bed not found: ID does not exist"
Jul 21 14:20:29.160615 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160540    2569 scope.go:117] "RemoveContainer" containerID="b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"
Jul 21 14:20:29.160747 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160726    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c"} err="failed to get container status \"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c\": rpc error: code = NotFound desc = could not find container \"b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c\": container with ID starting with b88e89c294a80032731c32b45491719301b166d350875aef0f5a9f67f73ca32c not found: ID does not exist"
Jul 21 14:20:29.160803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160749    2569 scope.go:117] "RemoveContainer" containerID="a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"
Jul 21 14:20:29.160985 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:29.160970    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682"} err="failed to get container status \"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682\": rpc error: code = NotFound desc = could not find container \"a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682\": container with ID starting with a7a91757784a4ae8c4276821910a2b69845366ea61ca450949646f9cf47cf682 not found: ID does not exist"
Jul 21 14:20:30.466753 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:30.466722    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" path="/var/lib/kubelet/pods/c04e637f-06e2-415e-8feb-94f8534faa0a/volumes"
Jul 21 14:20:31.627333 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627297    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:20:31.627774 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627642    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="prometheus" containerID="cri-o://577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5" gracePeriod=600
Jul 21 14:20:31.627774 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627661    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy" containerID="cri-o://e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df" gracePeriod=600
Jul 21 14:20:31.627774 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627716    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-thanos" containerID="cri-o://536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9" gracePeriod=600
Jul 21 14:20:31.627774 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627734    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="thanos-sidecar" containerID="cri-o://1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376" gracePeriod=600
Jul 21 14:20:31.627774 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627752    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-web" containerID="cri-o://bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6" gracePeriod=600
Jul 21 14:20:31.628009 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.627737    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="config-reloader" containerID="cri-o://f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc" gracePeriod=600
Jul 21 14:20:31.867043 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.867013    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:31.880279 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880223    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-tls\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880279 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880251    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880279 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880274    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-web-config\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880291    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-tls-assets\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880314    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-grpc-tls\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880328    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-config\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880348    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftf7x\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-kube-api-access-ftf7x\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880375    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-config-out\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880400    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-metrics-client-certs\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880448    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-1\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880471    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-2\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.880500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880498    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-metrics-client-ca\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880530    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880556    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-trusted-ca-bundle\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880582    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-db\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-kubelet-serving-ca-bundle\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880651    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-serving-certs-ca-bundle\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880677    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-0\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-thanos-prometheus-http-client-file\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.881095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.880762    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-kube-rbac-proxy\") pod \"e05b263b-f90b-4763-ae00-917f34a14226\" (UID: \"e05b263b-f90b-4763-ae00-917f34a14226\") "
Jul 21 14:20:31.882121 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.882090    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.882962 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.882868    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.883057 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.882977    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.883164 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.883122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.883915 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.883849    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.884589 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.884547    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.884669 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.884639    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.884731 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.884703    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:20:31.884814 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.884765    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.884880 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.884787    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:20:31.884972 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.884948    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:31.885172 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.885143    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.885249 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.885179    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-config-out" (OuterVolumeSpecName: "config-out") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:20:31.885475 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.885410    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-kube-api-access-ftf7x" (OuterVolumeSpecName: "kube-api-access-ftf7x") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "kube-api-access-ftf7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:20:31.885591 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.885563    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-config" (OuterVolumeSpecName: "config") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.885694 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.885616    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.885898 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.885877    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.886508 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.886464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.887840 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.887814    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.894024 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.894003    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-web-config" (OuterVolumeSpecName: "web-config") pod "e05b263b-f90b-4763-ae00-917f34a14226" (UID: "e05b263b-f90b-4763-ae00-917f34a14226"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:20:31.973932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.973897    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d9bfc4bbc-9ctl2"]
Jul 21 14:20:31.974156 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974143    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="thanos-sidecar"
Jul 21 14:20:31.974200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974158    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="thanos-sidecar"
Jul 21 14:20:31.974200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974170    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-web"
Jul 21 14:20:31.974200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974176    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-web"
Jul 21 14:20:31.974200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974187    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="prometheus"
Jul 21 14:20:31.974200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974197    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="prometheus"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974205    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-thanos"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974210    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-thanos"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974219    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974224    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974230    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-web"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974234    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-web"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974244    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="alertmanager"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974248    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="alertmanager"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974253    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="prom-label-proxy"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974257    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="prom-label-proxy"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974269    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="init-config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974279    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="init-config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974285    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974290    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974297    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974302    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974308    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974312    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974318    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e8e65e-7808-492a-84c2-b50b11edabfc" containerName="console"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974323    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e8e65e-7808-492a-84c2-b50b11edabfc" containerName="console"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974328    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-metric"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974334    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-metric"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974339    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="init-config-reloader"
Jul 21 14:20:31.974342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974345    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="init-config-reloader"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974386    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="thanos-sidecar"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974394    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e8e65e-7808-492a-84c2-b50b11edabfc" containerName="console"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974398    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-metric"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974403    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="config-reloader"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974408    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-web"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974412    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974435    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="prom-label-proxy"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974443    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="prometheus"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974448    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy-web"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974452    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="alertmanager"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974457    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05b263b-f90b-4763-ae00-917f34a14226" containerName="kube-rbac-proxy-thanos"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974462    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="config-reloader"
Jul 21 14:20:31.975003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.974467    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04e637f-06e2-415e-8feb-94f8534faa0a" containerName="kube-rbac-proxy"
Jul 21 14:20:31.977530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.977512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:31.979639 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.979574    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 14:20:31.979770 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.979657    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-br6h4\""
Jul 21 14:20:31.979870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.979846    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 14:20:31.979992 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.979888    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 14:20:31.979992 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.979851    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 14:20:31.980147 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.980033    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 14:20:31.980147 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.980037    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 14:20:31.980366 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.980352    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 14:20:31.981395 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981379    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981512 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981398    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-kube-rbac-proxy\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981512 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981408    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981512 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981433    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981512 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981505    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-web-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981516    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-tls-assets\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981530    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-grpc-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981539    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981547    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftf7x\" (UniqueName: \"kubernetes.io/projected/e05b263b-f90b-4763-ae00-917f34a14226-kube-api-access-ftf7x\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981556    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-config-out\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981564    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-metrics-client-certs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981576    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981589    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981602    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-metrics-client-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981615    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e05b263b-f90b-4763-ae00-917f34a14226-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981630    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981644    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-db\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981658    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981674    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.981803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.981688    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e05b263b-f90b-4763-ae00-917f34a14226-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:20:31.985178 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.985159    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 14:20:31.989308 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:31.989289    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d9bfc4bbc-9ctl2"]
Jul 21 14:20:32.082552 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082513    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-service-ca\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.082552 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-serving-cert\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.082802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-trusted-ca-bundle\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.082802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-oauth-serving-cert\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.082802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-oauth-config\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.082802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrzf\" (UniqueName: \"kubernetes.io/projected/ac1334df-2991-4daa-aa56-43a53bad7809-kube-api-access-ncrzf\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.082802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.082700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-console-config\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.117496 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117463    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9" exitCode=0
Jul 21 14:20:32.117496 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117491    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df" exitCode=0
Jul 21 14:20:32.117496 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117499    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6" exitCode=0
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117506    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376" exitCode=0
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117514    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc" exitCode=0
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117520    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05b263b-f90b-4763-ae00-917f34a14226" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5" exitCode=0
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"}
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117595    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"}
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117663    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"}
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"}
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117706    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.117729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"}
Jul 21 14:20:32.118118 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"}
Jul 21 14:20:32.118118 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.117764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e05b263b-f90b-4763-ae00-917f34a14226","Type":"ContainerDied","Data":"7f2d014cd23f2012aa64d70ad6c65023299b835b24df6a9b251a83912a729e1b"}
Jul 21 14:20:32.125490 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.125388    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.135990 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.135970    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.142946 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.142818    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.144667 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.144638    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:20:32.149069 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.149046    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:20:32.150445 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.150405    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.156979 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.156962    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.164279 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.164260    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.171172 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.171153    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.171449 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.171409    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.171534 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.171462    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"} err="failed to get container status \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist"
Jul 21 14:20:32.171534 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.171488    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.171753 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.171736    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.171788 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.171761    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"} err="failed to get container status \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist"
Jul 21 14:20:32.171788 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.171778    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.171995 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.171979    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.172034 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172002    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"} err="failed to get container status \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist"
Jul 21 14:20:32.172034 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172022    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.172221 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.172202    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.172278 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172228    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"} err="failed to get container status \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist"
Jul 21 14:20:32.172278 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172249    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.172509 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.172488    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.172578 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172514    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"} err="failed to get container status \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist"
Jul 21 14:20:32.172578 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172527    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.172782 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.172765    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.172825 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172786    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"} err="failed to get container status \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist"
Jul 21 14:20:32.172825 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.172801    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.173016 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:20:32.173000    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.173071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173023    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"} err="failed to get container status \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist"
Jul 21 14:20:32.173071 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173043    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.173296 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173276    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"} err="failed to get container status \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist"
Jul 21 14:20:32.173333 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173306    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.173552 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173536    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"} err="failed to get container status \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist"
Jul 21 14:20:32.173606 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173552    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.173764 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173748    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"} err="failed to get container status \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist"
Jul 21 14:20:32.173814 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173764    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.173952 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173924    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"} err="failed to get container status \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist"
Jul 21 14:20:32.173994 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.173953    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.174156 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174140    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"} err="failed to get container status \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist"
Jul 21 14:20:32.174196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174157    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.174360 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174345    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"} err="failed to get container status \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist"
Jul 21 14:20:32.174428 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174360    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.174608 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174592    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"} err="failed to get container status \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist"
Jul 21 14:20:32.174608 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174607    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.174816 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174801    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"} err="failed to get container status \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist"
Jul 21 14:20:32.174816 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.174816    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.175038 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175021    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"} err="failed to get container status \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist"
Jul 21 14:20:32.175079 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175047    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.175329 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175267    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"} err="failed to get container status \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist"
Jul 21 14:20:32.175329 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175310    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.176056 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175599    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"} err="failed to get container status \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist"
Jul 21 14:20:32.176056 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175623    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.176056 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.175986    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"} err="failed to get container status \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist"
Jul 21 14:20:32.176056 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176008    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.176311 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176287    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"} err="failed to get container status \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist"
Jul 21 14:20:32.176365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176316    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.176592 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176574    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"} err="failed to get container status \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist"
Jul 21 14:20:32.176657 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176595    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.176833 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176811    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"} err="failed to get container status \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist"
Jul 21 14:20:32.176833 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.176832    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.177058 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177041    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"} err="failed to get container status \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist"
Jul 21 14:20:32.177115 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177059    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.177168 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177110    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:20:32.177275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177257    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"} err="failed to get container status \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist"
Jul 21 14:20:32.177321 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177276    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.177579 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177550    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"} err="failed to get container status \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist"
Jul 21 14:20:32.177643 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177581    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.177823 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177807    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"} err="failed to get container status \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist"
Jul 21 14:20:32.177880 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.177823    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.178051 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178032    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"} err="failed to get container status \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist"
Jul 21 14:20:32.178051 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178051    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.178262 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178246    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"} err="failed to get container status \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist"
Jul 21 14:20:32.178307 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178263    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.178550 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178531    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"} err="failed to get container status \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist"
Jul 21 14:20:32.178607 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178552    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.178775 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178755    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"} err="failed to get container status \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist"
Jul 21 14:20:32.178835 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.178777    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.179057 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179033    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"} err="failed to get container status \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist"
Jul 21 14:20:32.179057 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179057    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.179335 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179314    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"} err="failed to get container status \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist"
Jul 21 14:20:32.179335 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179335    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.179569 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179550    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"} err="failed to get container status \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist"
Jul 21 14:20:32.179637 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179571    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.179799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179783    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"} err="failed to get container status \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist"
Jul 21 14:20:32.179846 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.179800    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.180019 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180002    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"} err="failed to get container status \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist"
Jul 21 14:20:32.180063 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180020    2569 scope.go:117] "RemoveContainer" containerID="536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"
Jul 21 14:20:32.180217 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180198    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9"} err="failed to get container status \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": rpc error: code = NotFound desc = could not find container \"536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9\": container with ID starting with 536d5b158c3826f82bfb58a1e97852fb3f42590a86cda52af55d846e92326ff9 not found: ID does not exist"
Jul 21 14:20:32.180217 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180216    2569 scope.go:117] "RemoveContainer" containerID="e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"
Jul 21 14:20:32.180435 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180403    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df"} err="failed to get container status \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": rpc error: code = NotFound desc = could not find container \"e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df\": container with ID starting with e014c11bb3c6e0f2a55c12fdf968dbb3b05d230f28439f0077810e22f8f8e2df not found: ID does not exist"
Jul 21 14:20:32.180503 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180436    2569 scope.go:117] "RemoveContainer" containerID="bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"
Jul 21 14:20:32.180645 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180630    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6"} err="failed to get container status \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": rpc error: code = NotFound desc = could not find container \"bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6\": container with ID starting with bb91c9337f716fd376ba290e2683cbb4976f11c1f12e0fef28f36a0e5423bdc6 not found: ID does not exist"
Jul 21 14:20:32.180718 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180647    2569 scope.go:117] "RemoveContainer" containerID="1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"
Jul 21 14:20:32.180828 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180809    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376"} err="failed to get container status \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": rpc error: code = NotFound desc = could not find container \"1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376\": container with ID starting with 1b6d26c24469d46d8ce5e032b03e9b0f2001f03e92f70454e52b32167797a376 not found: ID does not exist"
Jul 21 14:20:32.180874 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180829    2569 scope.go:117] "RemoveContainer" containerID="f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"
Jul 21 14:20:32.180986 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180969    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc"} err="failed to get container status \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": rpc error: code = NotFound desc = could not find container \"f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc\": container with ID starting with f7b8c154d6b64cd2c5d6ecc0c50693cbd9c9baa662f3259e3f1bd562a1772bfc not found: ID does not exist"
Jul 21 14:20:32.181054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.180988    2569 scope.go:117] "RemoveContainer" containerID="577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"
Jul 21 14:20:32.181157 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.181142    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5"} err="failed to get container status \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": rpc error: code = NotFound desc = could not find container \"577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5\": container with ID starting with 577096519d101baf5fc5b2138c9021872198755096c4796e5ee8e55566fccfd5 not found: ID does not exist"
Jul 21 14:20:32.181224 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.181159    2569 scope.go:117] "RemoveContainer" containerID="abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"
Jul 21 14:20:32.181394 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.181373    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8"} err="failed to get container status \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": rpc error: code = NotFound desc = could not find container \"abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8\": container with ID starting with abe0b03f469e749d5b00fbfe064c2d4dd2a5fa687b5b31a67c21f751928ac4a8 not found: ID does not exist"
Jul 21 14:20:32.181488 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.181474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.183261 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183241    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-service-ca\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.183357 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-serving-cert\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.183357 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-trusted-ca-bundle\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.183357 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-oauth-serving-cert\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.183530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183393    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-oauth-config\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.183530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183456    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrzf\" (UniqueName: \"kubernetes.io/projected/ac1334df-2991-4daa-aa56-43a53bad7809-kube-api-access-ncrzf\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.183530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.183485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-console-config\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.184059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184011    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 21 14:20:32.184059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184033    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 21 14:20:32.184309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184120    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 21 14:20:32.184309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-oauth-serving-cert\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.184309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184218    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-console-config\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.184309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-service-ca\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.184309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184309    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 21 14:20:32.184599 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184325    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 21 14:20:32.184599 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184346    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 21 14:20:32.184599 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184346    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 21 14:20:32.184599 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184515    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 21 14:20:32.184599 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184553    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 21 14:20:32.184816 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.184610    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-trusted-ca-bundle\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.185388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.185180    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f0dp7t9jbjrbj\""
Jul 21 14:20:32.185388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.185221    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 21 14:20:32.185388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.185262    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 21 14:20:32.185388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.185304    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 21 14:20:32.185388 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.185321    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gqskc\""
Jul 21 14:20:32.186372 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.186338    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-serving-cert\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.186819 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.186799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-oauth-config\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.187564 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.187548    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 21 14:20:32.190240 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.190220    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 21 14:20:32.193956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.193939    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrzf\" (UniqueName: \"kubernetes.io/projected/ac1334df-2991-4daa-aa56-43a53bad7809-kube-api-access-ncrzf\") pod \"console-6d9bfc4bbc-9ctl2\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") " pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.196590 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.196570    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:20:32.283856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.283856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ef10bfd-fd30-4abc-b014-702778f7470c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fccsm\" (UniqueName: \"kubernetes.io/projected/5ef10bfd-fd30-4abc-b014-702778f7470c-kube-api-access-fccsm\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283942    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.283980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ef10bfd-fd30-4abc-b014-702778f7470c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-config\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284115    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284156    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284223    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284254    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284323 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.284597 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.284340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.287851 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.287821    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:32.385653 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385622    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385653 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ef10bfd-fd30-4abc-b014-702778f7470c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fccsm\" (UniqueName: \"kubernetes.io/projected/5ef10bfd-fd30-4abc-b014-702778f7470c-kube-api-access-fccsm\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385715    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ef10bfd-fd30-4abc-b014-702778f7470c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-config\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385830    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385856    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.385900 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385913    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385960    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.385987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.386009    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.386032    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.386055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.386385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.386108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.387448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.388574    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ef10bfd-fd30-4abc-b014-702778f7470c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.388901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.389045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.389073    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.389252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ef10bfd-fd30-4abc-b014-702778f7470c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.389849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.389874    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390100 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.389982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.390640 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.390365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.391002    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.391282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.391668    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.392141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-config\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.392143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.392151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392579 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.392238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392805 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.392785    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ef10bfd-fd30-4abc-b014-702778f7470c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.392872 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.392855    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ef10bfd-fd30-4abc-b014-702778f7470c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.395190 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.395172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fccsm\" (UniqueName: \"kubernetes.io/projected/5ef10bfd-fd30-4abc-b014-702778f7470c-kube-api-access-fccsm\") pod \"prometheus-k8s-0\" (UID: \"5ef10bfd-fd30-4abc-b014-702778f7470c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.417470 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.417444    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d9bfc4bbc-9ctl2"]
Jul 21 14:20:32.420626 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:20:32.420603    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1334df_2991_4daa_aa56_43a53bad7809.slice/crio-920d05a7e29023d8a6a6d09a622349a64fd260a3418f049c7915fe5ebdd37f4c WatchSource:0}: Error finding container 920d05a7e29023d8a6a6d09a622349a64fd260a3418f049c7915fe5ebdd37f4c: Status 404 returned error can't find the container with id 920d05a7e29023d8a6a6d09a622349a64fd260a3418f049c7915fe5ebdd37f4c
Jul 21 14:20:32.467847 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.467792    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e05b263b-f90b-4763-ae00-917f34a14226" path="/var/lib/kubelet/pods/e05b263b-f90b-4763-ae00-917f34a14226/volumes"
Jul 21 14:20:32.492600 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.492575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:32.639191 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:32.639089    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 14:20:32.643136 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:20:32.643106    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef10bfd_fd30_4abc_b014_702778f7470c.slice/crio-df7fe8af9585d973ea13f794360b87cdd5346d715377d350d4d49e53fdd7cdc4 WatchSource:0}: Error finding container df7fe8af9585d973ea13f794360b87cdd5346d715377d350d4d49e53fdd7cdc4: Status 404 returned error can't find the container with id df7fe8af9585d973ea13f794360b87cdd5346d715377d350d4d49e53fdd7cdc4
Jul 21 14:20:33.123880 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:33.123842    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ef10bfd-fd30-4abc-b014-702778f7470c" containerID="68aa91bc5c1718ec4bbf057f1367fab178b705db9061a414c2a45772024a1052" exitCode=0
Jul 21 14:20:33.124076 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:33.123938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerDied","Data":"68aa91bc5c1718ec4bbf057f1367fab178b705db9061a414c2a45772024a1052"}
Jul 21 14:20:33.124076 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:33.124017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"df7fe8af9585d973ea13f794360b87cdd5346d715377d350d4d49e53fdd7cdc4"}
Jul 21 14:20:33.125458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:33.125437    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9bfc4bbc-9ctl2" event={"ID":"ac1334df-2991-4daa-aa56-43a53bad7809","Type":"ContainerStarted","Data":"c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c"}
Jul 21 14:20:33.125550 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:33.125472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9bfc4bbc-9ctl2" event={"ID":"ac1334df-2991-4daa-aa56-43a53bad7809","Type":"ContainerStarted","Data":"920d05a7e29023d8a6a6d09a622349a64fd260a3418f049c7915fe5ebdd37f4c"}
Jul 21 14:20:33.178231 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:33.178177    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d9bfc4bbc-9ctl2" podStartSLOduration=2.178156712 podStartE2EDuration="2.178156712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:20:33.176793385 +0000 UTC m=+159.311188760" watchObservedRunningTime="2026-07-21 14:20:33.178156712 +0000 UTC m=+159.312552086"
Jul 21 14:20:34.133611 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.133575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"30353cf55b2307ad5bb271598586164ef3fe73fc3258f223a17f92918c77eabf"}
Jul 21 14:20:34.134061 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.133694    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"748cdd89f9ab0aa1c77c34890544c1a827aff951335fb0ecd3cd393358dbe3a0"}
Jul 21 14:20:34.134061 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.133739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"e68687858407004cd4d29892e3d4c590f408ba2986549a5065ae767c8e2025d7"}
Jul 21 14:20:34.134061 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.133781    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"2fc73f8a083b0149cb37f6fb303611a6b0df3c0c9bc03a0334fdf53af922ccab"}
Jul 21 14:20:34.134061 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.133817    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"6a6a84f5cf623f4741ed6c2e16cbe87a6179dad996d8ff8c15e754ce3ad16ab4"}
Jul 21 14:20:34.134061 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.133855    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ef10bfd-fd30-4abc-b014-702778f7470c","Type":"ContainerStarted","Data":"071eca8db0b2e15fcd19000e304799bef6be8620d73fbe009b49dbd437049c9d"}
Jul 21 14:20:34.164951 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:34.164880    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.144165577 podStartE2EDuration="2.164861183s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.695606ms" isStatefulPod=false podCreationTimestamp="2026-07-21 14:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:20:34.162992415 +0000 UTC m=+160.297387788" watchObservedRunningTime="2026-07-21 14:20:34.164861183 +0000 UTC m=+160.299256557"
Jul 21 14:20:37.493620 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:37.493581    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:20:42.287936 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:42.287902    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:42.288306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:42.287980    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:42.292641 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:42.292623    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:20:43.171247 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:20:43.171219    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:21:32.492935 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:21:32.492900    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:21:32.508600 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:21:32.508572    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:21:33.350876 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:21:33.350851    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 14:22:54.312536 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:22:54.312507    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 14:23:21.127554 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.127524    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7t2xd"]
Jul 21 14:23:21.130459 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.130438    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.132671 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.132651    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 14:23:21.132913 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.132898    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rjnvw\""
Jul 21 14:23:21.133291 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.133276    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 14:23:21.141592 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.141554    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7t2xd"]
Jul 21 14:23:21.215669 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.215637    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blw25\" (UniqueName: \"kubernetes.io/projected/3fc4bb42-4546-4f05-ab33-8b3542eefcf6-kube-api-access-blw25\") pod \"cert-manager-55b4d954d5-7t2xd\" (UID: \"3fc4bb42-4546-4f05-ab33-8b3542eefcf6\") " pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.215812 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.215675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc4bb42-4546-4f05-ab33-8b3542eefcf6-bound-sa-token\") pod \"cert-manager-55b4d954d5-7t2xd\" (UID: \"3fc4bb42-4546-4f05-ab33-8b3542eefcf6\") " pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.316856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.316821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc4bb42-4546-4f05-ab33-8b3542eefcf6-bound-sa-token\") pod \"cert-manager-55b4d954d5-7t2xd\" (UID: \"3fc4bb42-4546-4f05-ab33-8b3542eefcf6\") " pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.317036 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.316915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blw25\" (UniqueName: \"kubernetes.io/projected/3fc4bb42-4546-4f05-ab33-8b3542eefcf6-kube-api-access-blw25\") pod \"cert-manager-55b4d954d5-7t2xd\" (UID: \"3fc4bb42-4546-4f05-ab33-8b3542eefcf6\") " pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.326350 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.326325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc4bb42-4546-4f05-ab33-8b3542eefcf6-bound-sa-token\") pod \"cert-manager-55b4d954d5-7t2xd\" (UID: \"3fc4bb42-4546-4f05-ab33-8b3542eefcf6\") " pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.326542 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.326523    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blw25\" (UniqueName: \"kubernetes.io/projected/3fc4bb42-4546-4f05-ab33-8b3542eefcf6-kube-api-access-blw25\") pod \"cert-manager-55b4d954d5-7t2xd\" (UID: \"3fc4bb42-4546-4f05-ab33-8b3542eefcf6\") " pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.450311 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.450289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7t2xd"
Jul 21 14:23:21.578063 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.578031    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7t2xd"]
Jul 21 14:23:21.580908 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:23:21.580872    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fc4bb42_4546_4f05_ab33_8b3542eefcf6.slice/crio-95314462e41898dc933fecfd475311809c986fbda4bad5ff46f462580fa53ca7 WatchSource:0}: Error finding container 95314462e41898dc933fecfd475311809c986fbda4bad5ff46f462580fa53ca7: Status 404 returned error can't find the container with id 95314462e41898dc933fecfd475311809c986fbda4bad5ff46f462580fa53ca7
Jul 21 14:23:21.582739 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.582723    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:23:21.692781 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:21.692743    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7t2xd" event={"ID":"3fc4bb42-4546-4f05-ab33-8b3542eefcf6","Type":"ContainerStarted","Data":"95314462e41898dc933fecfd475311809c986fbda4bad5ff46f462580fa53ca7"}
Jul 21 14:23:24.712265 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:24.712225    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7t2xd" event={"ID":"3fc4bb42-4546-4f05-ab33-8b3542eefcf6","Type":"ContainerStarted","Data":"a1c95da805e59b9a1572b77b67faa500df7398c3219c56adf68e4bbf30471743"}
Jul 21 14:23:24.733895 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:24.733848    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7t2xd" podStartSLOduration=0.785803973 podStartE2EDuration="3.733828193s" totalImagesPullingTime="2.94802422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:24.732130002 +0000 UTC m=+330.866525377" watchObservedRunningTime="2026-07-21 14:23:24.733828193 +0000 UTC m=+330.868223568"
Jul 21 14:23:45.384924 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.384849    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"]
Jul 21 14:23:45.387927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.387908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.389880 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.389859    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 21 14:23:45.390443 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.390404    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 21 14:23:45.390538 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.390528    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nxv49\""
Jul 21 14:23:45.390631 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.390615    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 14:23:45.390714 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.390700    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 21 14:23:45.390818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.390804    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 14:23:45.405218 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.405191    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"]
Jul 21 14:23:45.517574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.517538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-cert\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.517739 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.517689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-metrics-cert\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.517972 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.517809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75jxx\" (UniqueName: \"kubernetes.io/projected/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-kube-api-access-75jxx\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.517972 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.517874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-manager-config\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.619106 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.619069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-cert\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.619287 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.619137    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-metrics-cert\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.619287 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.619172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75jxx\" (UniqueName: \"kubernetes.io/projected/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-kube-api-access-75jxx\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.619287 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.619191    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-manager-config\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.619957 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.619931    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-manager-config\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.621737 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.621715    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-cert\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.621841 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.621761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-metrics-cert\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.628472 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.628451    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75jxx\" (UniqueName: \"kubernetes.io/projected/4284e49c-0e0a-4a92-a17e-8c4d6f38245d-kube-api-access-75jxx\") pod \"lws-controller-manager-6df87d547c-gzps8\" (UID: \"4284e49c-0e0a-4a92-a17e-8c4d6f38245d\") " pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.696962 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.696938    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:45.841274 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.841237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"]
Jul 21 14:23:45.844223 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:23:45.844189    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4284e49c_0e0a_4a92_a17e_8c4d6f38245d.slice/crio-268ae6cfc97ffad615811fa677e2726b06efa22a977e0338df5c1c192e837639 WatchSource:0}: Error finding container 268ae6cfc97ffad615811fa677e2726b06efa22a977e0338df5c1c192e837639: Status 404 returned error can't find the container with id 268ae6cfc97ffad615811fa677e2726b06efa22a977e0338df5c1c192e837639
Jul 21 14:23:45.892917 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.892889    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"]
Jul 21 14:23:45.897184 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.897166    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:45.899971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.899950    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 14:23:45.900086 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.900051    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 14:23:45.900380 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.900350    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 14:23:45.900572 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.900556    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 14:23:45.900634 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.900618    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n8qqh\""
Jul 21 14:23:45.916281 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:45.916255    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"]
Jul 21 14:23:46.022772 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.022696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e662208-e323-4a0f-b7ad-8b70cc0c8818-manifests\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.022772 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.022732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e662208-e323-4a0f-b7ad-8b70cc0c8818-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.022772 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.022757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqxwh\" (UniqueName: \"kubernetes.io/projected/8e662208-e323-4a0f-b7ad-8b70cc0c8818-kube-api-access-fqxwh\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.023009 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.022799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e662208-e323-4a0f-b7ad-8b70cc0c8818-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.023009 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.022849    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e662208-e323-4a0f-b7ad-8b70cc0c8818-tmp\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.124057 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.124021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e662208-e323-4a0f-b7ad-8b70cc0c8818-tmp\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.124217 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.124081    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e662208-e323-4a0f-b7ad-8b70cc0c8818-manifests\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.124217 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.124198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e662208-e323-4a0f-b7ad-8b70cc0c8818-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.124320 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.124231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqxwh\" (UniqueName: \"kubernetes.io/projected/8e662208-e323-4a0f-b7ad-8b70cc0c8818-kube-api-access-fqxwh\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.124320 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.124265    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e662208-e323-4a0f-b7ad-8b70cc0c8818-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.124537 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.124503    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e662208-e323-4a0f-b7ad-8b70cc0c8818-manifests\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.126659 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.126629    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e662208-e323-4a0f-b7ad-8b70cc0c8818-tmp\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.126754 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.126671    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e662208-e323-4a0f-b7ad-8b70cc0c8818-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.126754 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.126695    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e662208-e323-4a0f-b7ad-8b70cc0c8818-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.136115 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.136097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqxwh\" (UniqueName: \"kubernetes.io/projected/8e662208-e323-4a0f-b7ad-8b70cc0c8818-kube-api-access-fqxwh\") pod \"opendatahub-operator-controller-manager-86bb54d755-wrxfj\" (UID: \"8e662208-e323-4a0f-b7ad-8b70cc0c8818\") " pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.207901 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.207859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:46.341840 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.341805    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"]
Jul 21 14:23:46.346570 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:23:46.346547    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e662208_e323_4a0f_b7ad_8b70cc0c8818.slice/crio-217051c99976476787bbe642665e58a17411c96badceb2b8cf5788272e6149a8 WatchSource:0}: Error finding container 217051c99976476787bbe642665e58a17411c96badceb2b8cf5788272e6149a8: Status 404 returned error can't find the container with id 217051c99976476787bbe642665e58a17411c96badceb2b8cf5788272e6149a8
Jul 21 14:23:46.792615 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.792574    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj" event={"ID":"8e662208-e323-4a0f-b7ad-8b70cc0c8818","Type":"ContainerStarted","Data":"217051c99976476787bbe642665e58a17411c96badceb2b8cf5788272e6149a8"}
Jul 21 14:23:46.793939 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:46.793914    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8" event={"ID":"4284e49c-0e0a-4a92-a17e-8c4d6f38245d","Type":"ContainerStarted","Data":"268ae6cfc97ffad615811fa677e2726b06efa22a977e0338df5c1c192e837639"}
Jul 21 14:23:50.812115 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:50.812083    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8" event={"ID":"4284e49c-0e0a-4a92-a17e-8c4d6f38245d","Type":"ContainerStarted","Data":"3767c0a314e5c853a14d751ac91d14bf31a68f93531f26c898c1f9fb153af296"}
Jul 21 14:23:50.812552 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:50.812231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:23:50.813580 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:50.813558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj" event={"ID":"8e662208-e323-4a0f-b7ad-8b70cc0c8818","Type":"ContainerStarted","Data":"1ca5c48db9cecfad2f39587590d8ae2fbb0d6498a706f23fd1e9b97351b810d0"}
Jul 21 14:23:50.836394 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:50.836224    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8" podStartSLOduration=1.709970796 podStartE2EDuration="5.836204885s" totalImagesPullingTime="4.126234089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:50.835980314 +0000 UTC m=+356.970375689" watchObservedRunningTime="2026-07-21 14:23:50.836204885 +0000 UTC m=+356.970600250"
Jul 21 14:23:51.818895 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:51.818859    2569 generic.go:356] "Generic (PLEG): container finished" podID="8e662208-e323-4a0f-b7ad-8b70cc0c8818" containerID="1ca5c48db9cecfad2f39587590d8ae2fbb0d6498a706f23fd1e9b97351b810d0" exitCode=0
Jul 21 14:23:51.819359 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:51.818950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj" event={"ID":"8e662208-e323-4a0f-b7ad-8b70cc0c8818","Type":"ContainerDied","Data":"1ca5c48db9cecfad2f39587590d8ae2fbb0d6498a706f23fd1e9b97351b810d0"}
Jul 21 14:23:52.825106 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:52.825072    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj" event={"ID":"8e662208-e323-4a0f-b7ad-8b70cc0c8818","Type":"ContainerStarted","Data":"caedb2607e8caff9cadc8db8ed8802afb96f598682cc7ab8dbcfc638bb1f5c33"}
Jul 21 14:23:52.825524 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:52.825188    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:23:52.849467 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:23:52.849398    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj" podStartSLOduration=2.766904237 podStartE2EDuration="7.84938107s" totalImagesPullingTime="4.057123691s" totalInitContainerRuntime="1.025353142s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:52.848467751 +0000 UTC m=+358.982863140" watchObservedRunningTime="2026-07-21 14:23:52.84938107 +0000 UTC m=+358.983776444"
Jul 21 14:24:01.821599 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:01.821570    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6df87d547c-gzps8"
Jul 21 14:24:03.831340 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:03.831315    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86bb54d755-wrxfj"
Jul 21 14:24:06.758675 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.758635    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bbd655967-thj96"]
Jul 21 14:24:06.767569 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.767543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.769790 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.769766    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 14:24:06.770081 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.770061    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 14:24:06.770306 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.770289    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 14:24:06.770686 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.770661    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 14:24:06.770795 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.770667    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cfb9n\""
Jul 21 14:24:06.779270 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.779243    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bbd655967-thj96"]
Jul 21 14:24:06.800245 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.800212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc629773-d072-453c-86ef-b5843a33fb3f-tls-certs\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.800382 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.800271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4ljs\" (UniqueName: \"kubernetes.io/projected/dc629773-d072-453c-86ef-b5843a33fb3f-kube-api-access-m4ljs\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.800382 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.800293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc629773-d072-453c-86ef-b5843a33fb3f-tmp\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.901378 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.901343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc629773-d072-453c-86ef-b5843a33fb3f-tls-certs\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.901574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.901462    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4ljs\" (UniqueName: \"kubernetes.io/projected/dc629773-d072-453c-86ef-b5843a33fb3f-kube-api-access-m4ljs\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.901574 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.901532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc629773-d072-453c-86ef-b5843a33fb3f-tmp\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.904337 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.904305    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc629773-d072-453c-86ef-b5843a33fb3f-tmp\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.904673 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.904650    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc629773-d072-453c-86ef-b5843a33fb3f-tls-certs\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:06.919152 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:06.919123    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4ljs\" (UniqueName: \"kubernetes.io/projected/dc629773-d072-453c-86ef-b5843a33fb3f-kube-api-access-m4ljs\") pod \"kube-auth-proxy-5bbd655967-thj96\" (UID: \"dc629773-d072-453c-86ef-b5843a33fb3f\") " pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:07.077800 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:07.077716    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96"
Jul 21 14:24:07.219167 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:07.219134    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bbd655967-thj96"]
Jul 21 14:24:07.222089 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:24:07.222062    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc629773_d072_453c_86ef_b5843a33fb3f.slice/crio-ddb12563f616ff708ede1b45d18cb69e8db69a16ad4c9929f5e1be6bb9477e1a WatchSource:0}: Error finding container ddb12563f616ff708ede1b45d18cb69e8db69a16ad4c9929f5e1be6bb9477e1a: Status 404 returned error can't find the container with id ddb12563f616ff708ede1b45d18cb69e8db69a16ad4c9929f5e1be6bb9477e1a
Jul 21 14:24:07.881681 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:07.881644    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96" event={"ID":"dc629773-d072-453c-86ef-b5843a33fb3f","Type":"ContainerStarted","Data":"ddb12563f616ff708ede1b45d18cb69e8db69a16ad4c9929f5e1be6bb9477e1a"}
Jul 21 14:24:10.899355 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:10.899319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96" event={"ID":"dc629773-d072-453c-86ef-b5843a33fb3f","Type":"ContainerStarted","Data":"f1e38a96107de06554d23d7e2c29cf31ca4a7a6e1f16e2d865db148fa79b64e9"}
Jul 21 14:24:10.919030 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:10.918975    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bbd655967-thj96" podStartSLOduration=2.133578199 podStartE2EDuration="4.918955084s" totalImagesPullingTime="2.785376885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:24:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:24:10.918061357 +0000 UTC m=+377.052456731" watchObservedRunningTime="2026-07-21 14:24:10.918955084 +0000 UTC m=+377.053350460"
Jul 21 14:24:18.608984 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.608953    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"]
Jul 21 14:24:18.614592 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.613462    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:18.616138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.616118    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5rtt2\""
Jul 21 14:24:18.616329 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.616308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 14:24:18.624947 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.624920    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"]
Jul 21 14:24:18.696791 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.696751    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:18.696791 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.696796    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r4rw\" (UniqueName: \"kubernetes.io/projected/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-kube-api-access-6r4rw\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:18.798140 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.798108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:18.798301 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.798155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r4rw\" (UniqueName: \"kubernetes.io/projected/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-kube-api-access-6r4rw\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:18.798301 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:24:18.798257    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 21 14:24:18.798372 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:24:18.798346    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-cert podName:40c27eae-2b98-4af8-bc7b-7ae6a83b4529 nodeName:}" failed. No retries permitted until 2026-07-21 14:24:19.298328683 +0000 UTC m=+385.432724035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-cert") pod "llmisvc-controller-manager-777c6bd7d5-wwjvz" (UID: "40c27eae-2b98-4af8-bc7b-7ae6a83b4529") : secret "llmisvc-webhook-server-cert" not found
Jul 21 14:24:18.808002 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:18.807965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r4rw\" (UniqueName: \"kubernetes.io/projected/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-kube-api-access-6r4rw\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:19.302320 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:19.302291    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:19.304795 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:19.304765    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c27eae-2b98-4af8-bc7b-7ae6a83b4529-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wwjvz\" (UID: \"40c27eae-2b98-4af8-bc7b-7ae6a83b4529\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:19.526705 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:19.526666    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:19.673072 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:19.673043    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"]
Jul 21 14:24:19.676083 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:24:19.676057    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod40c27eae_2b98_4af8_bc7b_7ae6a83b4529.slice/crio-b6e37c6be7b17d6d23551e612c453f7d0ce352270d9ab8738bf30360015fd610 WatchSource:0}: Error finding container b6e37c6be7b17d6d23551e612c453f7d0ce352270d9ab8738bf30360015fd610: Status 404 returned error can't find the container with id b6e37c6be7b17d6d23551e612c453f7d0ce352270d9ab8738bf30360015fd610
Jul 21 14:24:19.932447 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:19.932347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz" event={"ID":"40c27eae-2b98-4af8-bc7b-7ae6a83b4529","Type":"ContainerStarted","Data":"b6e37c6be7b17d6d23551e612c453f7d0ce352270d9ab8738bf30360015fd610"}
Jul 21 14:24:23.949047 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:23.949012    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz" event={"ID":"40c27eae-2b98-4af8-bc7b-7ae6a83b4529","Type":"ContainerStarted","Data":"a4bdf4790ff57144a1c2bf9a6474f829c3aa841ffa7594f0e04597cdf1cc0112"}
Jul 21 14:24:23.949456 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:23.949125    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:24:23.968228 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:23.968181    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz" podStartSLOduration=2.670305963 podStartE2EDuration="5.968166356s" totalImagesPullingTime="3.297860393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:24:23.967509197 +0000 UTC m=+390.101904584" watchObservedRunningTime="2026-07-21 14:24:23.968166356 +0000 UTC m=+390.102561730"
Jul 21 14:24:31.859248 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.859211    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cc6bdcc65-vw7rz"]
Jul 21 14:24:31.861701 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.861684    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.873885 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.873861    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc6bdcc65-vw7rz"]
Jul 21 14:24:31.915228 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk97n\" (UniqueName: \"kubernetes.io/projected/fe1609b1-aa49-493c-90e3-eff174844f8b-kube-api-access-tk97n\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.915376 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-service-ca\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.915376 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe1609b1-aa49-493c-90e3-eff174844f8b-console-oauth-config\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.915376 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-oauth-serving-cert\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.915529 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-console-config\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.915529 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-trusted-ca-bundle\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:31.915529 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:31.915472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe1609b1-aa49-493c-90e3-eff174844f8b-console-serving-cert\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016253 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe1609b1-aa49-493c-90e3-eff174844f8b-console-serving-cert\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016474 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk97n\" (UniqueName: \"kubernetes.io/projected/fe1609b1-aa49-493c-90e3-eff174844f8b-kube-api-access-tk97n\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016474 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-service-ca\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016474 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016387    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe1609b1-aa49-493c-90e3-eff174844f8b-console-oauth-config\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016474 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-oauth-serving-cert\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016694 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-console-config\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.016694 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.016529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-trusted-ca-bundle\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.017202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.017174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-service-ca\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.017307 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.017174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-oauth-serving-cert\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.017367 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.017324    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-trusted-ca-bundle\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.017530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.017505    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe1609b1-aa49-493c-90e3-eff174844f8b-console-config\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.018872 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.018853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe1609b1-aa49-493c-90e3-eff174844f8b-console-serving-cert\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.018974 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.018875    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe1609b1-aa49-493c-90e3-eff174844f8b-console-oauth-config\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.025971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.025946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk97n\" (UniqueName: \"kubernetes.io/projected/fe1609b1-aa49-493c-90e3-eff174844f8b-kube-api-access-tk97n\") pod \"console-cc6bdcc65-vw7rz\" (UID: \"fe1609b1-aa49-493c-90e3-eff174844f8b\") " pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.170343 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.170247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:32.302303 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.302272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc6bdcc65-vw7rz"]
Jul 21 14:24:32.305103 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:24:32.305074    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1609b1_aa49_493c_90e3_eff174844f8b.slice/crio-d5f838ad1e0d9b94aa43fc2c896a750ee67545103009cfad1f6252e3bdb2dbfc WatchSource:0}: Error finding container d5f838ad1e0d9b94aa43fc2c896a750ee67545103009cfad1f6252e3bdb2dbfc: Status 404 returned error can't find the container with id d5f838ad1e0d9b94aa43fc2c896a750ee67545103009cfad1f6252e3bdb2dbfc
Jul 21 14:24:32.983877 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.983841    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc6bdcc65-vw7rz" event={"ID":"fe1609b1-aa49-493c-90e3-eff174844f8b","Type":"ContainerStarted","Data":"1efa2713260743caedfaaa35b11962e95be64247a160dff4a3c3feff3c5c82fa"}
Jul 21 14:24:32.984249 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:32.983892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc6bdcc65-vw7rz" event={"ID":"fe1609b1-aa49-493c-90e3-eff174844f8b","Type":"ContainerStarted","Data":"d5f838ad1e0d9b94aa43fc2c896a750ee67545103009cfad1f6252e3bdb2dbfc"}
Jul 21 14:24:33.006658 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:33.006608    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cc6bdcc65-vw7rz" podStartSLOduration=2.006575222 podStartE2EDuration="2.006575222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:24:33.0050851 +0000 UTC m=+399.139480475" watchObservedRunningTime="2026-07-21 14:24:33.006575222 +0000 UTC m=+399.140970595"
Jul 21 14:24:42.170631 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:42.170579    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:42.171052 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:42.170649    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:42.175661 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:42.175640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:43.026532 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:43.026501    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cc6bdcc65-vw7rz"
Jul 21 14:24:43.078155 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:43.078120    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d9bfc4bbc-9ctl2"]
Jul 21 14:24:54.956183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:24:54.956144    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wwjvz"
Jul 21 14:25:08.099185 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.099142    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d9bfc4bbc-9ctl2" podUID="ac1334df-2991-4daa-aa56-43a53bad7809" containerName="console" containerID="cri-o://c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c" gracePeriod=15
Jul 21 14:25:08.342292 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.342270    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:25:08.424270 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424197    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-trusted-ca-bundle\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424270 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424266    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-oauth-serving-cert\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424489 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424290    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-oauth-config\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424489 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424338    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-console-config\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424489 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424369    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-serving-cert\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424489 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424401    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-service-ca\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424489 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncrzf\" (UniqueName: \"kubernetes.io/projected/ac1334df-2991-4daa-aa56-43a53bad7809-kube-api-access-ncrzf\") pod \"ac1334df-2991-4daa-aa56-43a53bad7809\" (UID: \"ac1334df-2991-4daa-aa56-43a53bad7809\") "
Jul 21 14:25:08.424729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424674    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:25:08.424784 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424744    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:25:08.424838 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424820    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-console-config" (OuterVolumeSpecName: "console-config") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:25:08.424882 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.424830    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:25:08.426753 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.426722    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac1334df-2991-4daa-aa56-43a53bad7809-kube-api-access-ncrzf" (OuterVolumeSpecName: "kube-api-access-ncrzf") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "kube-api-access-ncrzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:25:08.426861 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.426765    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:25:08.426861 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.426778    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac1334df-2991-4daa-aa56-43a53bad7809" (UID: "ac1334df-2991-4daa-aa56-43a53bad7809"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:25:08.525763 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525724    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-oauth-serving-cert\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:08.525763 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525757    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-oauth-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:08.525971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525796    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-console-config\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:08.525971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525810    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac1334df-2991-4daa-aa56-43a53bad7809-console-serving-cert\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:08.525971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525823    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:08.525971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525834    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncrzf\" (UniqueName: \"kubernetes.io/projected/ac1334df-2991-4daa-aa56-43a53bad7809-kube-api-access-ncrzf\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:08.525971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:08.525846    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac1334df-2991-4daa-aa56-43a53bad7809-trusted-ca-bundle\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:25:09.121010 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.120972    2569 generic.go:356] "Generic (PLEG): container finished" podID="ac1334df-2991-4daa-aa56-43a53bad7809" containerID="c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c" exitCode=0
Jul 21 14:25:09.121458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.121018    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9bfc4bbc-9ctl2"
Jul 21 14:25:09.121458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.121055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9bfc4bbc-9ctl2" event={"ID":"ac1334df-2991-4daa-aa56-43a53bad7809","Type":"ContainerDied","Data":"c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c"}
Jul 21 14:25:09.121458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.121112    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9bfc4bbc-9ctl2" event={"ID":"ac1334df-2991-4daa-aa56-43a53bad7809","Type":"ContainerDied","Data":"920d05a7e29023d8a6a6d09a622349a64fd260a3418f049c7915fe5ebdd37f4c"}
Jul 21 14:25:09.121458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.121134    2569 scope.go:117] "RemoveContainer" containerID="c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c"
Jul 21 14:25:09.129409 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.129392    2569 scope.go:117] "RemoveContainer" containerID="c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c"
Jul 21 14:25:09.129714 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:25:09.129690    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c\": container with ID starting with c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c not found: ID does not exist" containerID="c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c"
Jul 21 14:25:09.129799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.129719    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c"} err="failed to get container status \"c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c\": rpc error: code = NotFound desc = could not find container \"c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c\": container with ID starting with c77e3133afe1405a281af03cadd6278a9362a1728b2dbae3c813f146cf37e92c not found: ID does not exist"
Jul 21 14:25:09.142409 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.142387    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d9bfc4bbc-9ctl2"]
Jul 21 14:25:09.146726 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:09.146708    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d9bfc4bbc-9ctl2"]
Jul 21 14:25:10.467851 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:25:10.467820    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac1334df-2991-4daa-aa56-43a53bad7809" path="/var/lib/kubelet/pods/ac1334df-2991-4daa-aa56-43a53bad7809/volumes"
Jul 21 14:26:01.374078 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.374044    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"]
Jul 21 14:26:01.374492 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.374325    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac1334df-2991-4daa-aa56-43a53bad7809" containerName="console"
Jul 21 14:26:01.374492 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.374335    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac1334df-2991-4daa-aa56-43a53bad7809" containerName="console"
Jul 21 14:26:01.374492 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.374391    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac1334df-2991-4daa-aa56-43a53bad7809" containerName="console"
Jul 21 14:26:01.378241 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.378219    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.380394 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.380370    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 14:26:01.380538 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.380430    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 21 14:26:01.380993 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.380971    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q6lsw\""
Jul 21 14:26:01.381091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.381006    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 21 14:26:01.381091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.381018    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 14:26:01.390328 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.390308    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"]
Jul 21 14:26:01.478186 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.478151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.478362 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.478202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.478362 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.478283    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d86kh\" (UniqueName: \"kubernetes.io/projected/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-kube-api-access-d86kh\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.579566 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.579482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.579566 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.579542    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.579749 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.579576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d86kh\" (UniqueName: \"kubernetes.io/projected/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-kube-api-access-d86kh\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.579749 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:26:01.579627    2569 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 21 14:26:01.579749 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:26:01.579704    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-plugin-serving-cert podName:4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7 nodeName:}" failed. No retries permitted until 2026-07-21 14:26:02.079687354 +0000 UTC m=+488.214082706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-v4c47" (UID: "4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7") : secret "plugin-serving-cert" not found
Jul 21 14:26:01.580207 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.580189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:01.591625 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:01.591592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d86kh\" (UniqueName: \"kubernetes.io/projected/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-kube-api-access-d86kh\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:02.084271 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:02.084233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:02.086757 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:02.086735    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v4c47\" (UID: \"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:02.287758 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:02.287718    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"
Jul 21 14:26:02.428076 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:02.428051    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47"]
Jul 21 14:26:02.430225 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:26:02.430198    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c19610a_f74d_40bd_bdd6_3f3f7f1f2ff7.slice/crio-ec3e6689da9ee30b330e69279562dba3a9d1cef30f36a9af5748f9cf65b70814 WatchSource:0}: Error finding container ec3e6689da9ee30b330e69279562dba3a9d1cef30f36a9af5748f9cf65b70814: Status 404 returned error can't find the container with id ec3e6689da9ee30b330e69279562dba3a9d1cef30f36a9af5748f9cf65b70814
Jul 21 14:26:03.332687 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:03.332652    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47" event={"ID":"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7","Type":"ContainerStarted","Data":"ec3e6689da9ee30b330e69279562dba3a9d1cef30f36a9af5748f9cf65b70814"}
Jul 21 14:26:27.460001 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:27.459962    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47" event={"ID":"4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7","Type":"ContainerStarted","Data":"87882e3bfc0e0d620f6512a23552619846e03ca0e8aa48ade9c455d226d7ae6d"}
Jul 21 14:26:27.485932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:27.485876    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v4c47" podStartSLOduration=2.553130657 podStartE2EDuration="26.485859986s" totalImagesPullingTime="23.932729329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:26:27.485576669 +0000 UTC m=+513.619972043" watchObservedRunningTime="2026-07-21 14:26:27.485859986 +0000 UTC m=+513.620255360"
Jul 21 14:26:56.909196 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:56.909116    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:26:56.968745 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:56.968713    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:26:56.968745 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:56.968750    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:26:56.968939 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:56.968843    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:56.971023 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:56.971001    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 14:26:57.087765 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.087735    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cbcb661-8caf-47d0-b274-d9fff98edaf8-config-file\") pod \"limitador-limitador-54d6d988d5-4sj9s\" (UID: \"8cbcb661-8caf-47d0-b274-d9fff98edaf8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.087990 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.087779    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j852\" (UniqueName: \"kubernetes.io/projected/8cbcb661-8caf-47d0-b274-d9fff98edaf8-kube-api-access-9j852\") pod \"limitador-limitador-54d6d988d5-4sj9s\" (UID: \"8cbcb661-8caf-47d0-b274-d9fff98edaf8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.188576 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.188497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cbcb661-8caf-47d0-b274-d9fff98edaf8-config-file\") pod \"limitador-limitador-54d6d988d5-4sj9s\" (UID: \"8cbcb661-8caf-47d0-b274-d9fff98edaf8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.188576 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.188543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j852\" (UniqueName: \"kubernetes.io/projected/8cbcb661-8caf-47d0-b274-d9fff98edaf8-kube-api-access-9j852\") pod \"limitador-limitador-54d6d988d5-4sj9s\" (UID: \"8cbcb661-8caf-47d0-b274-d9fff98edaf8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.189120 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.189101    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cbcb661-8caf-47d0-b274-d9fff98edaf8-config-file\") pod \"limitador-limitador-54d6d988d5-4sj9s\" (UID: \"8cbcb661-8caf-47d0-b274-d9fff98edaf8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.197241 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.197220    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j852\" (UniqueName: \"kubernetes.io/projected/8cbcb661-8caf-47d0-b274-d9fff98edaf8-kube-api-access-9j852\") pod \"limitador-limitador-54d6d988d5-4sj9s\" (UID: \"8cbcb661-8caf-47d0-b274-d9fff98edaf8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.278395 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.278365    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:26:57.309651 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.309625    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mv9bt"]
Jul 21 14:26:57.312980 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.312960    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:26:57.315162 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.315140    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-596n2\""
Jul 21 14:26:57.321471 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.321180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv9bt"]
Jul 21 14:26:57.391007 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.390979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzl65\" (UniqueName: \"kubernetes.io/projected/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6-kube-api-access-wzl65\") pod \"authorino-58cd955675-mv9bt\" (UID: \"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6\") " pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:26:57.413557 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.413529    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:26:57.416832 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:26:57.416803    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbcb661_8caf_47d0_b274_d9fff98edaf8.slice/crio-5a35207508eb25267412ce1458e47e6fba537569d65dc837855b8f7cfac59bdf WatchSource:0}: Error finding container 5a35207508eb25267412ce1458e47e6fba537569d65dc837855b8f7cfac59bdf: Status 404 returned error can't find the container with id 5a35207508eb25267412ce1458e47e6fba537569d65dc837855b8f7cfac59bdf
Jul 21 14:26:57.492313 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.492279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzl65\" (UniqueName: \"kubernetes.io/projected/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6-kube-api-access-wzl65\") pod \"authorino-58cd955675-mv9bt\" (UID: \"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6\") " pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:26:57.505482 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.505453    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzl65\" (UniqueName: \"kubernetes.io/projected/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6-kube-api-access-wzl65\") pod \"authorino-58cd955675-mv9bt\" (UID: \"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6\") " pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:26:57.585804 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.585774    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s" event={"ID":"8cbcb661-8caf-47d0-b274-d9fff98edaf8","Type":"ContainerStarted","Data":"5a35207508eb25267412ce1458e47e6fba537569d65dc837855b8f7cfac59bdf"}
Jul 21 14:26:57.624019 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.623992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:26:57.751522 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:57.751436    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv9bt"]
Jul 21 14:26:57.754333 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:26:57.754308    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ad7841d_5d59_4724_a7ad_fe4069ba5cf6.slice/crio-159d9e590734ebc94f5a07e0925bdb2c7a0d587c9eb5a835725157c6352106a1 WatchSource:0}: Error finding container 159d9e590734ebc94f5a07e0925bdb2c7a0d587c9eb5a835725157c6352106a1: Status 404 returned error can't find the container with id 159d9e590734ebc94f5a07e0925bdb2c7a0d587c9eb5a835725157c6352106a1
Jul 21 14:26:58.592951 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:26:58.592896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv9bt" event={"ID":"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6","Type":"ContainerStarted","Data":"159d9e590734ebc94f5a07e0925bdb2c7a0d587c9eb5a835725157c6352106a1"}
Jul 21 14:27:01.824604 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:01.824569    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv9bt"]
Jul 21 14:27:02.613661 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.613625    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s" event={"ID":"8cbcb661-8caf-47d0-b274-d9fff98edaf8","Type":"ContainerStarted","Data":"1982f3c7a01737418f3fa9e6db8ad8bb55bfb9d54537a6eff8fc3189b1929b46"}
Jul 21 14:27:02.613853 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.613715    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:27:02.615084 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.615057    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv9bt" event={"ID":"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6","Type":"ContainerStarted","Data":"4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97"}
Jul 21 14:27:02.615193 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.615126    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mv9bt" podUID="4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" containerName="authorino" containerID="cri-o://4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97" gracePeriod=30
Jul 21 14:27:02.631054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.631013    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s" podStartSLOduration=2.222910684 podStartE2EDuration="6.630997285s" totalImagesPullingTime="4.408086601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:27:02.630769453 +0000 UTC m=+548.765164828" watchObservedRunningTime="2026-07-21 14:27:02.630997285 +0000 UTC m=+548.765392660"
Jul 21 14:27:02.647095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.647049    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mv9bt" podStartSLOduration=1.568441099 podStartE2EDuration="5.647033253s" totalImagesPullingTime="4.078592154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:27:02.645569614 +0000 UTC m=+548.779964989" watchObservedRunningTime="2026-07-21 14:27:02.647033253 +0000 UTC m=+548.781428627"
Jul 21 14:27:02.865775 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.865714    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:27:02.944335 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.944302    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzl65\" (UniqueName: \"kubernetes.io/projected/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6-kube-api-access-wzl65\") pod \"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6\" (UID: \"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6\") "
Jul 21 14:27:02.946689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:02.946650    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6-kube-api-access-wzl65" (OuterVolumeSpecName: "kube-api-access-wzl65") pod "4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" (UID: "4ad7841d-5d59-4724-a7ad-fe4069ba5cf6"). InnerVolumeSpecName "kube-api-access-wzl65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:27:03.045722 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.045691    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzl65\" (UniqueName: \"kubernetes.io/projected/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6-kube-api-access-wzl65\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:27:03.620286 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.620256    2569 generic.go:356] "Generic (PLEG): container finished" podID="4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" containerID="4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97" exitCode=0
Jul 21 14:27:03.620506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.620311    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv9bt"
Jul 21 14:27:03.620506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.620347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv9bt" event={"ID":"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6","Type":"ContainerDied","Data":"4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97"}
Jul 21 14:27:03.620506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.620396    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv9bt" event={"ID":"4ad7841d-5d59-4724-a7ad-fe4069ba5cf6","Type":"ContainerDied","Data":"159d9e590734ebc94f5a07e0925bdb2c7a0d587c9eb5a835725157c6352106a1"}
Jul 21 14:27:03.620506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.620444    2569 scope.go:117] "RemoveContainer" containerID="4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97"
Jul 21 14:27:03.630193 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.630176    2569 scope.go:117] "RemoveContainer" containerID="4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97"
Jul 21 14:27:03.630588 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:27:03.630567    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97\": container with ID starting with 4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97 not found: ID does not exist" containerID="4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97"
Jul 21 14:27:03.630662 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.630597    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97"} err="failed to get container status \"4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97\": rpc error: code = NotFound desc = could not find container \"4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97\": container with ID starting with 4411bd2c49386d105137d28cb52c15a62d7321fd93b6e6dec0b587b212937b97 not found: ID does not exist"
Jul 21 14:27:03.642003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.641976    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv9bt"]
Jul 21 14:27:03.647269 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:03.647247    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv9bt"]
Jul 21 14:27:04.467309 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:04.467281    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" path="/var/lib/kubelet/pods/4ad7841d-5d59-4724-a7ad-fe4069ba5cf6/volumes"
Jul 21 14:27:13.621234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:13.621208    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"
Jul 21 14:27:28.488867 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.488835    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s95n9"]
Jul 21 14:27:28.489473 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.489278    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" containerName="authorino"
Jul 21 14:27:28.489473 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.489295    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" containerName="authorino"
Jul 21 14:27:28.489473 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.489358    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad7841d-5d59-4724-a7ad-fe4069ba5cf6" containerName="authorino"
Jul 21 14:27:28.492526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.492507    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:28.494625 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.494605    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-596n2\""
Jul 21 14:27:28.498054 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.498034    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s95n9"]
Jul 21 14:27:28.551355 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.551319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk2m8\" (UniqueName: \"kubernetes.io/projected/8f4f46e7-33ac-4267-832d-7273124dc45e-kube-api-access-sk2m8\") pod \"authorino-5d4b67c446-s95n9\" (UID: \"8f4f46e7-33ac-4267-832d-7273124dc45e\") " pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:28.652189 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.652148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk2m8\" (UniqueName: \"kubernetes.io/projected/8f4f46e7-33ac-4267-832d-7273124dc45e-kube-api-access-sk2m8\") pod \"authorino-5d4b67c446-s95n9\" (UID: \"8f4f46e7-33ac-4267-832d-7273124dc45e\") " pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:28.661458 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.661437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk2m8\" (UniqueName: \"kubernetes.io/projected/8f4f46e7-33ac-4267-832d-7273124dc45e-kube-api-access-sk2m8\") pod \"authorino-5d4b67c446-s95n9\" (UID: \"8f4f46e7-33ac-4267-832d-7273124dc45e\") " pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:28.731770 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.731737    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s95n9"]
Jul 21 14:27:28.731988 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.731978    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:28.864818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.864794    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s95n9"]
Jul 21 14:27:28.867287 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:27:28.867260    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f4f46e7_33ac_4267_832d_7273124dc45e.slice/crio-1bd5f770db59eb51115a74790906a26e778044a70763f35fbc2c4cde7b2687f3 WatchSource:0}: Error finding container 1bd5f770db59eb51115a74790906a26e778044a70763f35fbc2c4cde7b2687f3: Status 404 returned error can't find the container with id 1bd5f770db59eb51115a74790906a26e778044a70763f35fbc2c4cde7b2687f3
Jul 21 14:27:28.894929 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.894897    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f66d8b989-66wlg"]
Jul 21 14:27:28.903363 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.903331    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:28.905529 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.905505    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 21 14:27:28.906151 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.906128    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f66d8b989-66wlg"]
Jul 21 14:27:28.955475 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.955414    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5433b49d-7d97-4cc5-8655-5ea999d817f8-tls-cert\") pod \"authorino-f66d8b989-66wlg\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") " pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:28.955660 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:28.955573    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2z4s\" (UniqueName: \"kubernetes.io/projected/5433b49d-7d97-4cc5-8655-5ea999d817f8-kube-api-access-t2z4s\") pod \"authorino-f66d8b989-66wlg\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") " pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:29.056928 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.056832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2z4s\" (UniqueName: \"kubernetes.io/projected/5433b49d-7d97-4cc5-8655-5ea999d817f8-kube-api-access-t2z4s\") pod \"authorino-f66d8b989-66wlg\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") " pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:29.056928 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.056887    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5433b49d-7d97-4cc5-8655-5ea999d817f8-tls-cert\") pod \"authorino-f66d8b989-66wlg\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") " pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:29.059591 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.059566    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5433b49d-7d97-4cc5-8655-5ea999d817f8-tls-cert\") pod \"authorino-f66d8b989-66wlg\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") " pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:29.069264 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.069237    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2z4s\" (UniqueName: \"kubernetes.io/projected/5433b49d-7d97-4cc5-8655-5ea999d817f8-kube-api-access-t2z4s\") pod \"authorino-f66d8b989-66wlg\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") " pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:29.213513 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.213405    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:27:29.345806 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.345735    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f66d8b989-66wlg"]
Jul 21 14:27:29.349019 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:27:29.348995    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5433b49d_7d97_4cc5_8655_5ea999d817f8.slice/crio-b06e0717a068dbb31ba469ef53fbe388e31344ea82b75a0f41cdb06f7186e287 WatchSource:0}: Error finding container b06e0717a068dbb31ba469ef53fbe388e31344ea82b75a0f41cdb06f7186e287: Status 404 returned error can't find the container with id b06e0717a068dbb31ba469ef53fbe388e31344ea82b75a0f41cdb06f7186e287
Jul 21 14:27:29.726754 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.726726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s95n9" event={"ID":"8f4f46e7-33ac-4267-832d-7273124dc45e","Type":"ContainerStarted","Data":"1bd5f770db59eb51115a74790906a26e778044a70763f35fbc2c4cde7b2687f3"}
Jul 21 14:27:29.727882 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:29.727851    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f66d8b989-66wlg" event={"ID":"5433b49d-7d97-4cc5-8655-5ea999d817f8","Type":"ContainerStarted","Data":"b06e0717a068dbb31ba469ef53fbe388e31344ea82b75a0f41cdb06f7186e287"}
Jul 21 14:27:30.736354 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:30.736316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s95n9" event={"ID":"8f4f46e7-33ac-4267-832d-7273124dc45e","Type":"ContainerStarted","Data":"d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7"}
Jul 21 14:27:30.736837 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:30.736360    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-s95n9" podUID="8f4f46e7-33ac-4267-832d-7273124dc45e" containerName="authorino" containerID="cri-o://d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7" gracePeriod=30
Jul 21 14:27:30.737820 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:30.737798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f66d8b989-66wlg" event={"ID":"5433b49d-7d97-4cc5-8655-5ea999d817f8","Type":"ContainerStarted","Data":"2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195"}
Jul 21 14:27:30.753166 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:30.753123    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-s95n9" podStartSLOduration=1.357656065 podStartE2EDuration="2.753109102s" totalImagesPullingTime="1.395453037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:27:30.751268485 +0000 UTC m=+576.885663879" watchObservedRunningTime="2026-07-21 14:27:30.753109102 +0000 UTC m=+576.887504476"
Jul 21 14:27:30.766684 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:30.766642    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f66d8b989-66wlg" podStartSLOduration=2.437885987 podStartE2EDuration="2.766627876s" totalImagesPullingTime="328.741889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:27:30.765356213 +0000 UTC m=+576.899751609" watchObservedRunningTime="2026-07-21 14:27:30.766627876 +0000 UTC m=+576.901023250"
Jul 21 14:27:30.982225 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:30.982193    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:31.078234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.078145    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk2m8\" (UniqueName: \"kubernetes.io/projected/8f4f46e7-33ac-4267-832d-7273124dc45e-kube-api-access-sk2m8\") pod \"8f4f46e7-33ac-4267-832d-7273124dc45e\" (UID: \"8f4f46e7-33ac-4267-832d-7273124dc45e\") "
Jul 21 14:27:31.080356 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.080331    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f4f46e7-33ac-4267-832d-7273124dc45e-kube-api-access-sk2m8" (OuterVolumeSpecName: "kube-api-access-sk2m8") pod "8f4f46e7-33ac-4267-832d-7273124dc45e" (UID: "8f4f46e7-33ac-4267-832d-7273124dc45e"). InnerVolumeSpecName "kube-api-access-sk2m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:27:31.178999 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.178948    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk2m8\" (UniqueName: \"kubernetes.io/projected/8f4f46e7-33ac-4267-832d-7273124dc45e-kube-api-access-sk2m8\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:27:31.743114 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.743078    2569 generic.go:356] "Generic (PLEG): container finished" podID="8f4f46e7-33ac-4267-832d-7273124dc45e" containerID="d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7" exitCode=0
Jul 21 14:27:31.743622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.743120    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s95n9"
Jul 21 14:27:31.743622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.743169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s95n9" event={"ID":"8f4f46e7-33ac-4267-832d-7273124dc45e","Type":"ContainerDied","Data":"d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7"}
Jul 21 14:27:31.743622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.743203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s95n9" event={"ID":"8f4f46e7-33ac-4267-832d-7273124dc45e","Type":"ContainerDied","Data":"1bd5f770db59eb51115a74790906a26e778044a70763f35fbc2c4cde7b2687f3"}
Jul 21 14:27:31.743622 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.743220    2569 scope.go:117] "RemoveContainer" containerID="d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7"
Jul 21 14:27:31.751612 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.751592    2569 scope.go:117] "RemoveContainer" containerID="d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7"
Jul 21 14:27:31.751890 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:27:31.751871    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7\": container with ID starting with d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7 not found: ID does not exist" containerID="d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7"
Jul 21 14:27:31.751944 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.751902    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7"} err="failed to get container status \"d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7\": rpc error: code = NotFound desc = could not find container \"d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7\": container with ID starting with d54fa1e6a9a8f120fefda732cdf06e842a95df5a17cdd5975aff7020e01364c7 not found: ID does not exist"
Jul 21 14:27:31.764988 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.764958    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s95n9"]
Jul 21 14:27:31.770610 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:31.770588    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s95n9"]
Jul 21 14:27:32.467273 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:27:32.467241    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f4f46e7-33ac-4267-832d-7273124dc45e" path="/var/lib/kubelet/pods/8f4f46e7-33ac-4267-832d-7273124dc45e/volumes"
Jul 21 14:28:10.674669 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:28:10.674639    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:29:01.213348 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:01.213310    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:29:05.618238 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:05.618194    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:29:28.620210 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:28.620174    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:29:42.124485 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:42.124447    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:29:55.018109 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:55.018019    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:29:58.574030 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.573996    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"]
Jul 21 14:29:58.574430 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.574297    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f4f46e7-33ac-4267-832d-7273124dc45e" containerName="authorino"
Jul 21 14:29:58.574430 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.574307    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f4f46e7-33ac-4267-832d-7273124dc45e" containerName="authorino"
Jul 21 14:29:58.574430 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.574376    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f4f46e7-33ac-4267-832d-7273124dc45e" containerName="authorino"
Jul 21 14:29:58.577382 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.577363    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.580021 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.579999    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 21 14:29:58.580116 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.580016    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5rcx2\""
Jul 21 14:29:58.580116 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.580004    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 14:29:58.580231 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.580122    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 14:29:58.587940 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.587921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"]
Jul 21 14:29:58.650585 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.650562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.650737 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.650594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.650737 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.650622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b67941-17a2-4c98-9849-8f162039d05b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.650737 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.650729    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.650849 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.650790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.650849 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.650812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmjhq\" (UniqueName: \"kubernetes.io/projected/75b67941-17a2-4c98-9849-8f162039d05b-kube-api-access-lmjhq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.751720 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.751689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.751720 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.751718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.751932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.751738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b67941-17a2-4c98-9849-8f162039d05b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.751932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.751765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.751932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.751798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.751932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.751816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmjhq\" (UniqueName: \"kubernetes.io/projected/75b67941-17a2-4c98-9849-8f162039d05b-kube-api-access-lmjhq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.752154 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.752131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.752209 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.752153    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.752253 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.752204    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.754205 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.754183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b67941-17a2-4c98-9849-8f162039d05b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.754362 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.754345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b67941-17a2-4c98-9849-8f162039d05b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.761937 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.761916    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmjhq\" (UniqueName: \"kubernetes.io/projected/75b67941-17a2-4c98-9849-8f162039d05b-kube-api-access-lmjhq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-228x4\" (UID: \"75b67941-17a2-4c98-9849-8f162039d05b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:58.886695 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:58.886614    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:29:59.022347 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:59.022325    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"]
Jul 21 14:29:59.024878 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:29:59.024850    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75b67941_17a2_4c98_9849_8f162039d05b.slice/crio-2d2ef7152053780b81fc032df7c0fe9b9143c938df80372df908bf92fe0c8754 WatchSource:0}: Error finding container 2d2ef7152053780b81fc032df7c0fe9b9143c938df80372df908bf92fe0c8754: Status 404 returned error can't find the container with id 2d2ef7152053780b81fc032df7c0fe9b9143c938df80372df908bf92fe0c8754
Jul 21 14:29:59.026173 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:59.026155    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:29:59.347655 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:29:59.347622    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4" event={"ID":"75b67941-17a2-4c98-9849-8f162039d05b","Type":"ContainerStarted","Data":"2d2ef7152053780b81fc032df7c0fe9b9143c938df80372df908bf92fe0c8754"}
Jul 21 14:30:01.359478 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:01.359445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4" event={"ID":"75b67941-17a2-4c98-9849-8f162039d05b","Type":"ContainerStarted","Data":"a7a7644eaace633564af33e1ebb7820ccef4e6a3b2988b7bbc0e0fde890f0d87"}
Jul 21 14:30:01.382042 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:01.381999    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4" podStartSLOduration=2.111705275 podStartE2EDuration="3.381983986s" totalImagesPullingTime="1.270278711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:30:01.379951951 +0000 UTC m=+727.514347338" watchObservedRunningTime="2026-07-21 14:30:01.381983986 +0000 UTC m=+727.516379360"
Jul 21 14:30:08.886729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:08.886693    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:30:08.887130 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:08.886747    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:30:08.899038 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:08.899019    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:30:09.117753 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:09.117717    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:30:09.404152 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:30:09.404122    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-228x4"
Jul 21 14:31:02.000449 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.000396    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b9b8b88b8-fkhkz"]
Jul 21 14:31:02.003576 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.003556    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.015479 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.015453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b9b8b88b8-fkhkz"]
Jul 21 14:31:02.118874 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.118837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twq27\" (UniqueName: \"kubernetes.io/projected/08e2c2d2-70f6-448f-bd8c-049d34de961c-kube-api-access-twq27\") pod \"authorino-7b9b8b88b8-fkhkz\" (UID: \"08e2c2d2-70f6-448f-bd8c-049d34de961c\") " pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.118874 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.118870    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/08e2c2d2-70f6-448f-bd8c-049d34de961c-tls-cert\") pod \"authorino-7b9b8b88b8-fkhkz\" (UID: \"08e2c2d2-70f6-448f-bd8c-049d34de961c\") " pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.220358 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.220323    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/08e2c2d2-70f6-448f-bd8c-049d34de961c-tls-cert\") pod \"authorino-7b9b8b88b8-fkhkz\" (UID: \"08e2c2d2-70f6-448f-bd8c-049d34de961c\") " pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.220568 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.220447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twq27\" (UniqueName: \"kubernetes.io/projected/08e2c2d2-70f6-448f-bd8c-049d34de961c-kube-api-access-twq27\") pod \"authorino-7b9b8b88b8-fkhkz\" (UID: \"08e2c2d2-70f6-448f-bd8c-049d34de961c\") " pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.223105 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.223075    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/08e2c2d2-70f6-448f-bd8c-049d34de961c-tls-cert\") pod \"authorino-7b9b8b88b8-fkhkz\" (UID: \"08e2c2d2-70f6-448f-bd8c-049d34de961c\") " pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.230099 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.230074    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twq27\" (UniqueName: \"kubernetes.io/projected/08e2c2d2-70f6-448f-bd8c-049d34de961c-kube-api-access-twq27\") pod \"authorino-7b9b8b88b8-fkhkz\" (UID: \"08e2c2d2-70f6-448f-bd8c-049d34de961c\") " pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.313152 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.313067    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz"
Jul 21 14:31:02.471469 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.471437    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b9b8b88b8-fkhkz"]
Jul 21 14:31:02.475281 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:31:02.475250    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08e2c2d2_70f6_448f_bd8c_049d34de961c.slice/crio-4c41a074e56f81d3d05c43aa2ffc83980029203a0a8d7f67dceffe561f1e2878 WatchSource:0}: Error finding container 4c41a074e56f81d3d05c43aa2ffc83980029203a0a8d7f67dceffe561f1e2878: Status 404 returned error can't find the container with id 4c41a074e56f81d3d05c43aa2ffc83980029203a0a8d7f67dceffe561f1e2878
Jul 21 14:31:02.610652 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:02.610575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz" event={"ID":"08e2c2d2-70f6-448f-bd8c-049d34de961c","Type":"ContainerStarted","Data":"4c41a074e56f81d3d05c43aa2ffc83980029203a0a8d7f67dceffe561f1e2878"}
Jul 21 14:31:03.617931 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.617897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz" event={"ID":"08e2c2d2-70f6-448f-bd8c-049d34de961c","Type":"ContainerStarted","Data":"0b1e5e3a8e249eacaac5331c6ac3296c33f3002322e0cd37c5836a47b14bd433"}
Jul 21 14:31:03.643040 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.642961    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b9b8b88b8-fkhkz" podStartSLOduration=2.001981311 podStartE2EDuration="2.642938117s" totalImagesPullingTime="640.956806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:31:03.641240113 +0000 UTC m=+789.775635516" watchObservedRunningTime="2026-07-21 14:31:03.642938117 +0000 UTC m=+789.777333494"
Jul 21 14:31:03.678559 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.678527    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f66d8b989-66wlg"]
Jul 21 14:31:03.678771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.678727    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-f66d8b989-66wlg" podUID="5433b49d-7d97-4cc5-8655-5ea999d817f8" containerName="authorino" containerID="cri-o://2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195" gracePeriod=30
Jul 21 14:31:03.938469 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.938444    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:31:03.943760 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.943738    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2z4s\" (UniqueName: \"kubernetes.io/projected/5433b49d-7d97-4cc5-8655-5ea999d817f8-kube-api-access-t2z4s\") pod \"5433b49d-7d97-4cc5-8655-5ea999d817f8\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") "
Jul 21 14:31:03.943894 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.943825    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5433b49d-7d97-4cc5-8655-5ea999d817f8-tls-cert\") pod \"5433b49d-7d97-4cc5-8655-5ea999d817f8\" (UID: \"5433b49d-7d97-4cc5-8655-5ea999d817f8\") "
Jul 21 14:31:03.946162 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.946130    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5433b49d-7d97-4cc5-8655-5ea999d817f8-kube-api-access-t2z4s" (OuterVolumeSpecName: "kube-api-access-t2z4s") pod "5433b49d-7d97-4cc5-8655-5ea999d817f8" (UID: "5433b49d-7d97-4cc5-8655-5ea999d817f8"). InnerVolumeSpecName "kube-api-access-t2z4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:31:03.954557 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:03.954526    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5433b49d-7d97-4cc5-8655-5ea999d817f8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5433b49d-7d97-4cc5-8655-5ea999d817f8" (UID: "5433b49d-7d97-4cc5-8655-5ea999d817f8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:31:04.044509 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.044475    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5433b49d-7d97-4cc5-8655-5ea999d817f8-tls-cert\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:31:04.044509 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.044509    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2z4s\" (UniqueName: \"kubernetes.io/projected/5433b49d-7d97-4cc5-8655-5ea999d817f8-kube-api-access-t2z4s\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:31:04.624208 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.624166    2569 generic.go:356] "Generic (PLEG): container finished" podID="5433b49d-7d97-4cc5-8655-5ea999d817f8" containerID="2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195" exitCode=0
Jul 21 14:31:04.624707 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.624243    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f66d8b989-66wlg"
Jul 21 14:31:04.624707 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.624261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f66d8b989-66wlg" event={"ID":"5433b49d-7d97-4cc5-8655-5ea999d817f8","Type":"ContainerDied","Data":"2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195"}
Jul 21 14:31:04.624707 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.624312    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f66d8b989-66wlg" event={"ID":"5433b49d-7d97-4cc5-8655-5ea999d817f8","Type":"ContainerDied","Data":"b06e0717a068dbb31ba469ef53fbe388e31344ea82b75a0f41cdb06f7186e287"}
Jul 21 14:31:04.624707 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.624335    2569 scope.go:117] "RemoveContainer" containerID="2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195"
Jul 21 14:31:04.634233 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.634215    2569 scope.go:117] "RemoveContainer" containerID="2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195"
Jul 21 14:31:04.634511 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:31:04.634492    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195\": container with ID starting with 2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195 not found: ID does not exist" containerID="2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195"
Jul 21 14:31:04.634570 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.634521    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195"} err="failed to get container status \"2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195\": rpc error: code = NotFound desc = could not find container \"2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195\": container with ID starting with 2947b1338ef6fac1dd32b6e3e79d7da4b5231acf1afde35da4fbe323c2956195 not found: ID does not exist"
Jul 21 14:31:04.641522 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.641501    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f66d8b989-66wlg"]
Jul 21 14:31:04.651028 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:04.651002    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f66d8b989-66wlg"]
Jul 21 14:31:06.468231 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:06.468197    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5433b49d-7d97-4cc5-8655-5ea999d817f8" path="/var/lib/kubelet/pods/5433b49d-7d97-4cc5-8655-5ea999d817f8/volumes"
Jul 21 14:31:31.321082 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:31.320998    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:31:40.725391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:40.725356    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:31:50.628806 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:31:50.628767    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:33:33.619397 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:33:33.619365    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:33:41.817276 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:33:41.817240    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:34:15.425899 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:34:15.425810    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:34:22.223436 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:34:22.223391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:34:32.716254 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:34:32.716216    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:34:38.516236 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:34:38.516199    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:34:48.024799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:34:48.024763    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:34:58.728795 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:34:58.728756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:35:10.219814 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:35:10.219778    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:35:20.021709 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:35:20.021672    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:35:42.614889 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:35:42.614857    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:35:51.223680 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:35:51.223599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:36:15.125032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:36:15.124996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:36:25.314288 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:36:25.314247    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:36:34.018091 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:36:34.018054    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:36:44.430717 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:36:44.430683    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:36:53.667465 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:36:53.667409    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:37:04.018928 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:37:04.018890    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:37:12.828230 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:37:12.828188    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:37:22.115860 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:37:22.115777    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:39:13.524519 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:39:13.524448    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:39:22.517105 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:39:22.517072    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:39:37.120645 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:39:37.120598    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:39:46.621746 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:39:46.621710    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:39:57.223253 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:39:57.223216    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:40:08.430537 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:40:08.430503    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:40:15.546138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:40:15.546056    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:40:25.420978 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:40:25.420941    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:40:32.926072 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:40:32.926022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:40:43.914967 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:40:43.914928    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:40:55.536489 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:40:55.536456    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:41:02.924522 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:02.924486    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:41:24.034121 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:24.034087    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:41:31.420389 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:31.420351    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:41:44.166893 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.166842    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"]
Jul 21 14:41:44.167813 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.167322    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5433b49d-7d97-4cc5-8655-5ea999d817f8" containerName="authorino"
Jul 21 14:41:44.167813 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.167341    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5433b49d-7d97-4cc5-8655-5ea999d817f8" containerName="authorino"
Jul 21 14:41:44.167813 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.167434    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5433b49d-7d97-4cc5-8655-5ea999d817f8" containerName="authorino"
Jul 21 14:41:44.170460 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.170441    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.172619 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.172588    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e128bfd9-kserve-self-signed-certs\""
Jul 21 14:41:44.189722 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.189698    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"]
Jul 21 14:41:44.282650 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.282619    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-dshm\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.282818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.282664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2twfs\" (UniqueName: \"kubernetes.io/projected/9d5b32d0-3c79-4d86-8747-1f357605c376-kube-api-access-2twfs\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.282818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.282682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-home\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.282818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.282774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-model-cache\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.282818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.282800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-tmp-dir\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.282992 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.282831    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b32d0-3c79-4d86-8747-1f357605c376-tls-certs\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.383918 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.383844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2twfs\" (UniqueName: \"kubernetes.io/projected/9d5b32d0-3c79-4d86-8747-1f357605c376-kube-api-access-2twfs\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.383918 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.383882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-home\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.383944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-model-cache\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.383968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-tmp-dir\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.384009    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b32d0-3c79-4d86-8747-1f357605c376-tls-certs\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.384065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-dshm\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384443 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.384398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-tmp-dir\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.384399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-model-cache\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.384528 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.384472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-home\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.386412 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.386395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-dshm\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.386607 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.386588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b32d0-3c79-4d86-8747-1f357605c376-tls-certs\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.394249 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.394224    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2twfs\" (UniqueName: \"kubernetes.io/projected/9d5b32d0-3c79-4d86-8747-1f357605c376-kube-api-access-2twfs\") pod \"e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") " pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.479719 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.479681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:44.625759 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:44.625724    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"]
Jul 21 14:41:44.629403 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:41:44.629379    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d5b32d0_3c79_4d86_8747_1f357605c376.slice/crio-4a57fa9b6f93a82bb0a382cd1dd131b4d5005aef18e926f007a71c8177c3f7f3 WatchSource:0}: Error finding container 4a57fa9b6f93a82bb0a382cd1dd131b4d5005aef18e926f007a71c8177c3f7f3: Status 404 returned error can't find the container with id 4a57fa9b6f93a82bb0a382cd1dd131b4d5005aef18e926f007a71c8177c3f7f3
Jul 21 14:41:45.307607 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:45.307567    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht" event={"ID":"9d5b32d0-3c79-4d86-8747-1f357605c376","Type":"ContainerStarted","Data":"90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15"}
Jul 21 14:41:45.307994 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:45.307617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht" event={"ID":"9d5b32d0-3c79-4d86-8747-1f357605c376","Type":"ContainerStarted","Data":"4a57fa9b6f93a82bb0a382cd1dd131b4d5005aef18e926f007a71c8177c3f7f3"}
Jul 21 14:41:45.338470 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:45.338402    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht" podStartSLOduration=1.338384665 podStartE2EDuration="1.338384665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:41:45.338236769 +0000 UTC m=+1431.472632142" watchObservedRunningTime="2026-07-21 14:41:45.338384665 +0000 UTC m=+1431.472780084"
Jul 21 14:41:46.764033 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:46.763997    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:41:54.480414 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:54.480387    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:54.480802 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:54.480448    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:54.493219 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:54.493196    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:41:55.362728 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:41:55.362695    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:42:12.369288 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.369251    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"]
Jul 21 14:42:12.372787 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.372770    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.375868 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.375849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e128bfd9-kserve-self-signed-certs\""
Jul 21 14:42:12.386096 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.386072    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"]
Jul 21 14:42:12.541448 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.541389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278d5be8-f1f3-45da-a22a-93c07d37ce55-tls-certs\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.541612 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.541474    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-home\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.541612 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.541491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-model-cache\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.541612 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.541532    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-tmp-dir\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.541716 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.541607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-dshm\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.541716 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.541658    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w8ft\" (UniqueName: \"kubernetes.io/projected/278d5be8-f1f3-45da-a22a-93c07d37ce55-kube-api-access-6w8ft\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.642525 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642410    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w8ft\" (UniqueName: \"kubernetes.io/projected/278d5be8-f1f3-45da-a22a-93c07d37ce55-kube-api-access-6w8ft\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.642525 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278d5be8-f1f3-45da-a22a-93c07d37ce55-tls-certs\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.642525 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-home\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.642843 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-model-cache\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.642843 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-tmp-dir\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.642843 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-dshm\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.643002 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.642979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-home\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.643073 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.643010    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-model-cache\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.643073 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.643061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-tmp-dir\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.645044 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.645023    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-dshm\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.645308 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.645290    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278d5be8-f1f3-45da-a22a-93c07d37ce55-tls-certs\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.654028 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.653998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w8ft\" (UniqueName: \"kubernetes.io/projected/278d5be8-f1f3-45da-a22a-93c07d37ce55-kube-api-access-6w8ft\") pod \"e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") " pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.682580 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.682551    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:12.816088 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:12.816054    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"]
Jul 21 14:42:12.819673 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:42:12.819647    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod278d5be8_f1f3_45da_a22a_93c07d37ce55.slice/crio-f8cb6f08ac371777287d194b77f59c37a3d245cc2813c9a0b411ab01be0aab0e WatchSource:0}: Error finding container f8cb6f08ac371777287d194b77f59c37a3d245cc2813c9a0b411ab01be0aab0e: Status 404 returned error can't find the container with id f8cb6f08ac371777287d194b77f59c37a3d245cc2813c9a0b411ab01be0aab0e
Jul 21 14:42:13.428669 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:13.428631    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn" event={"ID":"278d5be8-f1f3-45da-a22a-93c07d37ce55","Type":"ContainerStarted","Data":"bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283"}
Jul 21 14:42:13.429072 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:13.428680    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn" event={"ID":"278d5be8-f1f3-45da-a22a-93c07d37ce55","Type":"ContainerStarted","Data":"f8cb6f08ac371777287d194b77f59c37a3d245cc2813c9a0b411ab01be0aab0e"}
Jul 21 14:42:13.453980 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:13.453917    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn" podStartSLOduration=1.453898468 podStartE2EDuration="1.453898468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:42:13.449174017 +0000 UTC m=+1459.583569392" watchObservedRunningTime="2026-07-21 14:42:13.453898468 +0000 UTC m=+1459.588293844"
Jul 21 14:42:15.729372 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:15.729332    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:42:22.682677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:22.682646    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:22.683068 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:22.682693    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:22.695524 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:22.695501    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:23.484414 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:23.484377    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:43.813891 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:43.813857    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"]
Jul 21 14:42:44.565801 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:44.565761    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"]
Jul 21 14:42:47.627848 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:47.627807    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:42:58.834725 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:58.834663    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht" podUID="9d5b32d0-3c79-4d86-8747-1f357605c376" containerName="main" containerID="cri-o://90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15" gracePeriod=45
Jul 21 14:42:59.079009 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.078986    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:42:59.166357 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166273    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b32d0-3c79-4d86-8747-1f357605c376-tls-certs\") pod \"9d5b32d0-3c79-4d86-8747-1f357605c376\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") "
Jul 21 14:42:59.166357 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166327    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-dshm\") pod \"9d5b32d0-3c79-4d86-8747-1f357605c376\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") "
Jul 21 14:42:59.166666 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166407    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-model-cache\") pod \"9d5b32d0-3c79-4d86-8747-1f357605c376\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") "
Jul 21 14:42:59.166666 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166450    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-home\") pod \"9d5b32d0-3c79-4d86-8747-1f357605c376\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") "
Jul 21 14:42:59.166666 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166476    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2twfs\" (UniqueName: \"kubernetes.io/projected/9d5b32d0-3c79-4d86-8747-1f357605c376-kube-api-access-2twfs\") pod \"9d5b32d0-3c79-4d86-8747-1f357605c376\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") "
Jul 21 14:42:59.166666 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166499    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-tmp-dir\") pod \"9d5b32d0-3c79-4d86-8747-1f357605c376\" (UID: \"9d5b32d0-3c79-4d86-8747-1f357605c376\") "
Jul 21 14:42:59.166858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166743    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-home" (OuterVolumeSpecName: "home") pod "9d5b32d0-3c79-4d86-8747-1f357605c376" (UID: "9d5b32d0-3c79-4d86-8747-1f357605c376"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.166858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166757    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d5b32d0-3c79-4d86-8747-1f357605c376" (UID: "9d5b32d0-3c79-4d86-8747-1f357605c376"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.166858 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.166798    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d5b32d0-3c79-4d86-8747-1f357605c376" (UID: "9d5b32d0-3c79-4d86-8747-1f357605c376"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.168694 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.168672    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-dshm" (OuterVolumeSpecName: "dshm") pod "9d5b32d0-3c79-4d86-8747-1f357605c376" (UID: "9d5b32d0-3c79-4d86-8747-1f357605c376"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.168800 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.168738    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5b32d0-3c79-4d86-8747-1f357605c376-kube-api-access-2twfs" (OuterVolumeSpecName: "kube-api-access-2twfs") pod "9d5b32d0-3c79-4d86-8747-1f357605c376" (UID: "9d5b32d0-3c79-4d86-8747-1f357605c376"). InnerVolumeSpecName "kube-api-access-2twfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:42:59.168800 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.168785    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5b32d0-3c79-4d86-8747-1f357605c376-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d5b32d0-3c79-4d86-8747-1f357605c376" (UID: "9d5b32d0-3c79-4d86-8747-1f357605c376"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:42:59.267541 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.267490    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-model-cache\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.267541 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.267535    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-home\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.267541 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.267544    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2twfs\" (UniqueName: \"kubernetes.io/projected/9d5b32d0-3c79-4d86-8747-1f357605c376-kube-api-access-2twfs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.267541 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.267554    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-tmp-dir\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.267810 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.267566    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b32d0-3c79-4d86-8747-1f357605c376-tls-certs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.267810 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.267575    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b32d0-3c79-4d86-8747-1f357605c376-dshm\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.587581 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.587433    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn" podUID="278d5be8-f1f3-45da-a22a-93c07d37ce55" containerName="main" containerID="cri-o://bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283" gracePeriod=45
Jul 21 14:42:59.633315 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.633281    2569 generic.go:356] "Generic (PLEG): container finished" podID="9d5b32d0-3c79-4d86-8747-1f357605c376" containerID="90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15" exitCode=0
Jul 21 14:42:59.633500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.633334    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"
Jul 21 14:42:59.633500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.633342    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht" event={"ID":"9d5b32d0-3c79-4d86-8747-1f357605c376","Type":"ContainerDied","Data":"90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15"}
Jul 21 14:42:59.633500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.633381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht" event={"ID":"9d5b32d0-3c79-4d86-8747-1f357605c376","Type":"ContainerDied","Data":"4a57fa9b6f93a82bb0a382cd1dd131b4d5005aef18e926f007a71c8177c3f7f3"}
Jul 21 14:42:59.633500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.633403    2569 scope.go:117] "RemoveContainer" containerID="90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15"
Jul 21 14:42:59.659736 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.659684    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"]
Jul 21 14:42:59.664296 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.664271    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e128bfd9-kserve-9f4d958dc-8psht"]
Jul 21 14:42:59.698248 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.698225    2569 scope.go:117] "RemoveContainer" containerID="90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15"
Jul 21 14:42:59.698617 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:42:59.698597    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15\": container with ID starting with 90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15 not found: ID does not exist" containerID="90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15"
Jul 21 14:42:59.698700 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.698627    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15"} err="failed to get container status \"90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15\": rpc error: code = NotFound desc = could not find container \"90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15\": container with ID starting with 90916f6951557f4e39e925f08f8e24eff3192c4b8ee4ca72ff0c6f451348eb15 not found: ID does not exist"
Jul 21 14:42:59.835597 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.835574    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:42:59.871696 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871665    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278d5be8-f1f3-45da-a22a-93c07d37ce55-tls-certs\") pod \"278d5be8-f1f3-45da-a22a-93c07d37ce55\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") "
Jul 21 14:42:59.871831 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871748    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-home\") pod \"278d5be8-f1f3-45da-a22a-93c07d37ce55\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") "
Jul 21 14:42:59.871831 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-model-cache\") pod \"278d5be8-f1f3-45da-a22a-93c07d37ce55\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") "
Jul 21 14:42:59.871920 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871850    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-tmp-dir\") pod \"278d5be8-f1f3-45da-a22a-93c07d37ce55\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") "
Jul 21 14:42:59.871971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871925    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w8ft\" (UniqueName: \"kubernetes.io/projected/278d5be8-f1f3-45da-a22a-93c07d37ce55-kube-api-access-6w8ft\") pod \"278d5be8-f1f3-45da-a22a-93c07d37ce55\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") "
Jul 21 14:42:59.871971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871940    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-home" (OuterVolumeSpecName: "home") pod "278d5be8-f1f3-45da-a22a-93c07d37ce55" (UID: "278d5be8-f1f3-45da-a22a-93c07d37ce55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.871971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.871955    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-dshm\") pod \"278d5be8-f1f3-45da-a22a-93c07d37ce55\" (UID: \"278d5be8-f1f3-45da-a22a-93c07d37ce55\") "
Jul 21 14:42:59.872112 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.872051    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-model-cache" (OuterVolumeSpecName: "model-cache") pod "278d5be8-f1f3-45da-a22a-93c07d37ce55" (UID: "278d5be8-f1f3-45da-a22a-93c07d37ce55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.872167 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.872154    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "278d5be8-f1f3-45da-a22a-93c07d37ce55" (UID: "278d5be8-f1f3-45da-a22a-93c07d37ce55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.872346 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.872280    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-home\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.872346 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.872303    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-model-cache\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.872346 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.872313    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-tmp-dir\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.873979 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.873950    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/278d5be8-f1f3-45da-a22a-93c07d37ce55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "278d5be8-f1f3-45da-a22a-93c07d37ce55" (UID: "278d5be8-f1f3-45da-a22a-93c07d37ce55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:42:59.874131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.874109    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-dshm" (OuterVolumeSpecName: "dshm") pod "278d5be8-f1f3-45da-a22a-93c07d37ce55" (UID: "278d5be8-f1f3-45da-a22a-93c07d37ce55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:42:59.874131 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.874122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/278d5be8-f1f3-45da-a22a-93c07d37ce55-kube-api-access-6w8ft" (OuterVolumeSpecName: "kube-api-access-6w8ft") pod "278d5be8-f1f3-45da-a22a-93c07d37ce55" (UID: "278d5be8-f1f3-45da-a22a-93c07d37ce55"). InnerVolumeSpecName "kube-api-access-6w8ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:42:59.973396 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.973354    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w8ft\" (UniqueName: \"kubernetes.io/projected/278d5be8-f1f3-45da-a22a-93c07d37ce55-kube-api-access-6w8ft\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.973396 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.973389    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/278d5be8-f1f3-45da-a22a-93c07d37ce55-dshm\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:42:59.973396 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:42:59.973401    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278d5be8-f1f3-45da-a22a-93c07d37ce55-tls-certs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:43:00.420375 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.420335    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:43:00.468393 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.468360    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5b32d0-3c79-4d86-8747-1f357605c376" path="/var/lib/kubelet/pods/9d5b32d0-3c79-4d86-8747-1f357605c376/volumes"
Jul 21 14:43:00.638694 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.638655    2569 generic.go:356] "Generic (PLEG): container finished" podID="278d5be8-f1f3-45da-a22a-93c07d37ce55" containerID="bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283" exitCode=0
Jul 21 14:43:00.638870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.638702    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"
Jul 21 14:43:00.638870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.638744    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn" event={"ID":"278d5be8-f1f3-45da-a22a-93c07d37ce55","Type":"ContainerDied","Data":"bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283"}
Jul 21 14:43:00.638870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.638785    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn" event={"ID":"278d5be8-f1f3-45da-a22a-93c07d37ce55","Type":"ContainerDied","Data":"f8cb6f08ac371777287d194b77f59c37a3d245cc2813c9a0b411ab01be0aab0e"}
Jul 21 14:43:00.638870 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.638805    2569 scope.go:117] "RemoveContainer" containerID="bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283"
Jul 21 14:43:00.646834 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.646817    2569 scope.go:117] "RemoveContainer" containerID="bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283"
Jul 21 14:43:00.647104 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:43:00.647085    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283\": container with ID starting with bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283 not found: ID does not exist" containerID="bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283"
Jul 21 14:43:00.647152 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.647113    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283"} err="failed to get container status \"bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283\": rpc error: code = NotFound desc = could not find container \"bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283\": container with ID starting with bf141c98bdd92fded1f6db7f8a217ba1be2eb1ed8b064367493c2fc628b62283 not found: ID does not exist"
Jul 21 14:43:00.654728 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.654706    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"]
Jul 21 14:43:00.658136 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:00.658114    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e128bfd9-kserve-66844bd895-rqjfn"]
Jul 21 14:43:02.468244 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:02.468211    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="278d5be8-f1f3-45da-a22a-93c07d37ce55" path="/var/lib/kubelet/pods/278d5be8-f1f3-45da-a22a-93c07d37ce55/volumes"
Jul 21 14:43:09.224518 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:09.224481    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:43:19.724717 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:19.724637    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:43:30.224170 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:30.224127    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:43:38.831526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:38.831492    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:43:50.119160 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:50.119124    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:43:58.911926 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:43:58.911888    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:44:07.813932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:44:07.813896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:44:21.721530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:44:21.721492    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:44:31.022211 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:44:31.022170    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:44:39.916099 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:44:39.916057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:44:51.712685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:44:51.712598    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:45:04.212891 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:45:04.212854    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:45:13.445324 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:45:13.445288    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:45:24.221793 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:45:24.221756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:45:34.522906 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:45:34.522870    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:45:46.116635 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:45:46.116597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:46:01.907289 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907254    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"]
Jul 21 14:46:01.907723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907590    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5b32d0-3c79-4d86-8747-1f357605c376" containerName="main"
Jul 21 14:46:01.907723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907602    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5b32d0-3c79-4d86-8747-1f357605c376" containerName="main"
Jul 21 14:46:01.907723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907622    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="278d5be8-f1f3-45da-a22a-93c07d37ce55" containerName="main"
Jul 21 14:46:01.907723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907628    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="278d5be8-f1f3-45da-a22a-93c07d37ce55" containerName="main"
Jul 21 14:46:01.907723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907681    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="278d5be8-f1f3-45da-a22a-93c07d37ce55" containerName="main"
Jul 21 14:46:01.907723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.907693    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5b32d0-3c79-4d86-8747-1f357605c376" containerName="main"
Jul 21 14:46:01.910789 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.910770    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:01.912862 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.912842    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-78a7b505-serving-cert\""
Jul 21 14:46:01.912994 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.912892    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:46:01.912994 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.912939    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:46:01.913384 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.913366    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:46:01.922530 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.922507    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"]
Jul 21 14:46:01.967746 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.967708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-openshift-service-ca\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:01.967946 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.967815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmz5h\" (UniqueName: \"kubernetes.io/projected/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-kube-api-access-jmz5h\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:01.967946 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:01.967847    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.069138 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.069100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-openshift-service-ca\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.069358 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.069177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmz5h\" (UniqueName: \"kubernetes.io/projected/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-kube-api-access-jmz5h\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.069358 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.069197    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.069358 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:46:02.069315    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-serving-cert: secret "maas-api-e2e-ait-78a7b505-serving-cert" not found
Jul 21 14:46:02.069579 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:46:02.069389    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls podName:890fd9dd-a9f4-4553-962e-9ee2f7896a8d nodeName:}" failed. No retries permitted until 2026-07-21 14:46:02.569366891 +0000 UTC m=+1688.703762259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls") pod "maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" (UID: "890fd9dd-a9f4-4553-962e-9ee2f7896a8d") : secret "maas-api-e2e-ait-78a7b505-serving-cert" not found
Jul 21 14:46:02.069903 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.069881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-openshift-service-ca\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.079803 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.079775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmz5h\" (UniqueName: \"kubernetes.io/projected/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-kube-api-access-jmz5h\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.575656 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.575619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.578205 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.578179    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls\") pod \"maas-api-e2e-ait-78a7b505-85759ff99c-56x2n\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.820862 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.820817    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:02.959752 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.959719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"]
Jul 21 14:46:02.963155 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:46:02.963129    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890fd9dd_a9f4_4553_962e_9ee2f7896a8d.slice/crio-8aac64591e179fbee0f0d3eb5a61f977027f15926f08edf2a744cc5b49d8ebe6 WatchSource:0}: Error finding container 8aac64591e179fbee0f0d3eb5a61f977027f15926f08edf2a744cc5b49d8ebe6: Status 404 returned error can't find the container with id 8aac64591e179fbee0f0d3eb5a61f977027f15926f08edf2a744cc5b49d8ebe6
Jul 21 14:46:02.964467 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:02.964449    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:46:03.419601 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:03.419566    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" event={"ID":"890fd9dd-a9f4-4553-962e-9ee2f7896a8d","Type":"ContainerStarted","Data":"8aac64591e179fbee0f0d3eb5a61f977027f15926f08edf2a744cc5b49d8ebe6"}
Jul 21 14:46:05.429663 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:05.429577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" event={"ID":"890fd9dd-a9f4-4553-962e-9ee2f7896a8d","Type":"ContainerStarted","Data":"4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159"}
Jul 21 14:46:05.430070 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:05.429720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:05.450194 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:05.450129    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" podStartSLOduration=2.242933742 podStartE2EDuration="4.450108889s" totalImagesPullingTime="2.207175147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:46:05.449516252 +0000 UTC m=+1691.583911630" watchObservedRunningTime="2026-07-21 14:46:05.450108889 +0000 UTC m=+1691.584504264"
Jul 21 14:46:09.341895 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:09.341859    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:46:11.445184 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:11.445154    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:38.142272 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.142238    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"]
Jul 21 14:46:38.142768 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.142473    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" podUID="890fd9dd-a9f4-4553-962e-9ee2f7896a8d" containerName="maas-api" containerID="cri-o://4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159" gracePeriod=30
Jul 21 14:46:38.390482 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.390460    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:38.487529 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.487502    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-openshift-service-ca\") pod \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") "
Jul 21 14:46:38.487720 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.487538    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls\") pod \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") "
Jul 21 14:46:38.487720 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.487601    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmz5h\" (UniqueName: \"kubernetes.io/projected/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-kube-api-access-jmz5h\") pod \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\" (UID: \"890fd9dd-a9f4-4553-962e-9ee2f7896a8d\") "
Jul 21 14:46:38.487916 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.487888    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "890fd9dd-a9f4-4553-962e-9ee2f7896a8d" (UID: "890fd9dd-a9f4-4553-962e-9ee2f7896a8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:46:38.489913 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.489884    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "890fd9dd-a9f4-4553-962e-9ee2f7896a8d" (UID: "890fd9dd-a9f4-4553-962e-9ee2f7896a8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:46:38.490020 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.489928    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-kube-api-access-jmz5h" (OuterVolumeSpecName: "kube-api-access-jmz5h") pod "890fd9dd-a9f4-4553-962e-9ee2f7896a8d" (UID: "890fd9dd-a9f4-4553-962e-9ee2f7896a8d"). InnerVolumeSpecName "kube-api-access-jmz5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:46:38.576178 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.576142    2569 generic.go:356] "Generic (PLEG): container finished" podID="890fd9dd-a9f4-4553-962e-9ee2f7896a8d" containerID="4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159" exitCode=0
Jul 21 14:46:38.576342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.576187    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"
Jul 21 14:46:38.576342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.576224    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" event={"ID":"890fd9dd-a9f4-4553-962e-9ee2f7896a8d","Type":"ContainerDied","Data":"4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159"}
Jul 21 14:46:38.576342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.576273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n" event={"ID":"890fd9dd-a9f4-4553-962e-9ee2f7896a8d","Type":"ContainerDied","Data":"8aac64591e179fbee0f0d3eb5a61f977027f15926f08edf2a744cc5b49d8ebe6"}
Jul 21 14:46:38.576342 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.576298    2569 scope.go:117] "RemoveContainer" containerID="4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159"
Jul 21 14:46:38.587099 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.587072    2569 scope.go:117] "RemoveContainer" containerID="4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159"
Jul 21 14:46:38.587552 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:46:38.587529    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159\": container with ID starting with 4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159 not found: ID does not exist" containerID="4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159"
Jul 21 14:46:38.587686 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.587559    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159"} err="failed to get container status \"4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159\": rpc error: code = NotFound desc = could not find container \"4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159\": container with ID starting with 4ab70f113784d02710545c4706c632fd30e214282d07f5191b1c8f32c7700159 not found: ID does not exist"
Jul 21 14:46:38.588938 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.588904    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:46:38.588938 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.588930    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:46:38.589084 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.588946    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmz5h\" (UniqueName: \"kubernetes.io/projected/890fd9dd-a9f4-4553-962e-9ee2f7896a8d-kube-api-access-jmz5h\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:46:38.605020 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.604994    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"]
Jul 21 14:46:38.610763 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:38.610744    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-78a7b505-85759ff99c-56x2n"]
Jul 21 14:46:40.468226 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:40.468191    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="890fd9dd-a9f4-4553-962e-9ee2f7896a8d" path="/var/lib/kubelet/pods/890fd9dd-a9f4-4553-962e-9ee2f7896a8d/volumes"
Jul 21 14:46:40.924761 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:40.924676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:46:46.415079 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.415028    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"]
Jul 21 14:46:46.415527 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.415355    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890fd9dd-a9f4-4553-962e-9ee2f7896a8d" containerName="maas-api"
Jul 21 14:46:46.415527 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.415366    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="890fd9dd-a9f4-4553-962e-9ee2f7896a8d" containerName="maas-api"
Jul 21 14:46:46.415527 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.415431    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="890fd9dd-a9f4-4553-962e-9ee2f7896a8d" containerName="maas-api"
Jul 21 14:46:46.420017 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.419998    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.422752 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.422726    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:46:46.422905 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.422887    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:46:46.422989 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.422903    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-98cb868b-serving-cert\""
Jul 21 14:46:46.423331 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.423315    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:46:46.433308 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.433284    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"]
Jul 21 14:46:46.452237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.452211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-maas-api-tls\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.452354 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.452336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtj9p\" (UniqueName: \"kubernetes.io/projected/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-kube-api-access-jtj9p\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.452434 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.452379    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-openshift-service-ca\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.553051 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.552967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-openshift-service-ca\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.553223 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.553111    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-maas-api-tls\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.553223 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.553193    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtj9p\" (UniqueName: \"kubernetes.io/projected/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-kube-api-access-jtj9p\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.553546 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.553528    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-openshift-service-ca\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.555667 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.555649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-maas-api-tls\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.563989 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.563963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtj9p\" (UniqueName: \"kubernetes.io/projected/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-kube-api-access-jtj9p\") pod \"maas-api-e2e-migrate-98cb868b-dfcfd879-glmld\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.729587 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.729552    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:46.871205 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:46.871173    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"]
Jul 21 14:46:46.875092 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:46:46.875064    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f1696f_1ef9_40fc_b569_effb8f4bb0b4.slice/crio-39ba0dcf3df4e37dca1e3670f45171cdef79ba48b176feba9b0af76d9a7cb275 WatchSource:0}: Error finding container 39ba0dcf3df4e37dca1e3670f45171cdef79ba48b176feba9b0af76d9a7cb275: Status 404 returned error can't find the container with id 39ba0dcf3df4e37dca1e3670f45171cdef79ba48b176feba9b0af76d9a7cb275
Jul 21 14:46:47.622515 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:47.622478    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld" event={"ID":"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4","Type":"ContainerStarted","Data":"7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878"}
Jul 21 14:46:47.622931 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:47.622544    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:46:47.622931 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:47.622561    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld" event={"ID":"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4","Type":"ContainerStarted","Data":"39ba0dcf3df4e37dca1e3670f45171cdef79ba48b176feba9b0af76d9a7cb275"}
Jul 21 14:46:47.655043 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:47.654973    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld" podStartSLOduration=1.354419884 podStartE2EDuration="1.654953403s" totalImagesPullingTime="300.533519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:46:47.651480756 +0000 UTC m=+1733.785876130" watchObservedRunningTime="2026-07-21 14:46:47.654953403 +0000 UTC m=+1733.789348777"
Jul 21 14:46:52.543432 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:52.543385    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:46:53.632645 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:46:53.632618    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:47:22.089239 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.089194    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"]
Jul 21 14:47:22.089828 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.089464    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld" podUID="c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" containerName="maas-api" containerID="cri-o://7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878" gracePeriod=30
Jul 21 14:47:22.323089 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.323067    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:47:22.443186 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.443156    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtj9p\" (UniqueName: \"kubernetes.io/projected/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-kube-api-access-jtj9p\") pod \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") "
Jul 21 14:47:22.443365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.443231    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-openshift-service-ca\") pod \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") "
Jul 21 14:47:22.443365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.443253    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-maas-api-tls\") pod \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\" (UID: \"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4\") "
Jul 21 14:47:22.443603 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.443579    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" (UID: "c9f1696f-1ef9-40fc-b569-effb8f4bb0b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:47:22.445411 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.445386    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" (UID: "c9f1696f-1ef9-40fc-b569-effb8f4bb0b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:47:22.445411 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.445384    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-kube-api-access-jtj9p" (OuterVolumeSpecName: "kube-api-access-jtj9p") pod "c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" (UID: "c9f1696f-1ef9-40fc-b569-effb8f4bb0b4"). InnerVolumeSpecName "kube-api-access-jtj9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:47:22.544564 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.544526    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtj9p\" (UniqueName: \"kubernetes.io/projected/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-kube-api-access-jtj9p\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:47:22.544564 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.544568    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:47:22.544756 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.544587    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:47:22.771836 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.771756    2569 generic.go:356] "Generic (PLEG): container finished" podID="c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" containerID="7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878" exitCode=0
Jul 21 14:47:22.771836 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.771813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld" event={"ID":"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4","Type":"ContainerDied","Data":"7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878"}
Jul 21 14:47:22.771836 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.771816    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"
Jul 21 14:47:22.772060 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.771878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld" event={"ID":"c9f1696f-1ef9-40fc-b569-effb8f4bb0b4","Type":"ContainerDied","Data":"39ba0dcf3df4e37dca1e3670f45171cdef79ba48b176feba9b0af76d9a7cb275"}
Jul 21 14:47:22.772060 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.771898    2569 scope.go:117] "RemoveContainer" containerID="7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878"
Jul 21 14:47:22.780695 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.780678    2569 scope.go:117] "RemoveContainer" containerID="7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878"
Jul 21 14:47:22.780952 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:47:22.780933    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878\": container with ID starting with 7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878 not found: ID does not exist" containerID="7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878"
Jul 21 14:47:22.781023 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.780960    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878"} err="failed to get container status \"7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878\": rpc error: code = NotFound desc = could not find container \"7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878\": container with ID starting with 7492998353aa46e705a39e96ef8a810de1644c94a24c9a62f35a66c61b67d878 not found: ID does not exist"
Jul 21 14:47:22.798032 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.798007    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"]
Jul 21 14:47:22.804901 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:22.804878    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-98cb868b-dfcfd879-glmld"]
Jul 21 14:47:24.467722 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:24.467685    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" path="/var/lib/kubelet/pods/c9f1696f-1ef9-40fc-b569-effb8f4bb0b4/volumes"
Jul 21 14:47:24.937683 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:24.937596    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:47:29.701022 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.700987    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"]
Jul 21 14:47:29.701391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.701283    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" containerName="maas-api"
Jul 21 14:47:29.701391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.701293    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" containerName="maas-api"
Jul 21 14:47:29.701391 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.701360    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f1696f-1ef9-40fc-b569-effb8f4bb0b4" containerName="maas-api"
Jul 21 14:47:29.703677 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.703661    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.712212 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.711381    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0a74b282-serving-cert\""
Jul 21 14:47:29.712212 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.711817    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:47:29.712212 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.712046    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:47:29.712563 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.712532    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:47:29.721773 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.721751    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"]
Jul 21 14:47:29.808314 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.808277    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b22f77-f0a8-494a-9cac-45076725869d-openshift-service-ca\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.808535 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.808326    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39b22f77-f0a8-494a-9cac-45076725869d-maas-api-tls\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.808535 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.808434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7thx\" (UniqueName: \"kubernetes.io/projected/39b22f77-f0a8-494a-9cac-45076725869d-kube-api-access-q7thx\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.909122 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.909089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7thx\" (UniqueName: \"kubernetes.io/projected/39b22f77-f0a8-494a-9cac-45076725869d-kube-api-access-q7thx\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.909296 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.909139    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b22f77-f0a8-494a-9cac-45076725869d-openshift-service-ca\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.909296 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.909166    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39b22f77-f0a8-494a-9cac-45076725869d-maas-api-tls\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.909875 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.909849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b22f77-f0a8-494a-9cac-45076725869d-openshift-service-ca\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.911630 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.911608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39b22f77-f0a8-494a-9cac-45076725869d-maas-api-tls\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:29.918728 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:29.918707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7thx\" (UniqueName: \"kubernetes.io/projected/39b22f77-f0a8-494a-9cac-45076725869d-kube-api-access-q7thx\") pod \"maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:30.015316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:30.015218    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:30.166369 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:30.166240    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"]
Jul 21 14:47:30.169289 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:47:30.169262    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b22f77_f0a8_494a_9cac_45076725869d.slice/crio-ec5c159a049e7988ffcb7f6c4008081de505652b647964bbe4a88a7f0343e09f WatchSource:0}: Error finding container ec5c159a049e7988ffcb7f6c4008081de505652b647964bbe4a88a7f0343e09f: Status 404 returned error can't find the container with id ec5c159a049e7988ffcb7f6c4008081de505652b647964bbe4a88a7f0343e09f
Jul 21 14:47:30.808437 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:30.808390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx" event={"ID":"39b22f77-f0a8-494a-9cac-45076725869d","Type":"ContainerStarted","Data":"8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437"}
Jul 21 14:47:30.808904 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:30.808474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx" event={"ID":"39b22f77-f0a8-494a-9cac-45076725869d","Type":"ContainerStarted","Data":"ec5c159a049e7988ffcb7f6c4008081de505652b647964bbe4a88a7f0343e09f"}
Jul 21 14:47:30.808904 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:30.808505    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:30.848922 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:30.848814    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx" podStartSLOduration=1.441012653 podStartE2EDuration="1.848796239s" totalImagesPullingTime="407.783586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:47:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:47:30.847901544 +0000 UTC m=+1776.982296931" watchObservedRunningTime="2026-07-21 14:47:30.848796239 +0000 UTC m=+1776.983191613"
Jul 21 14:47:36.817348 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:36.817312    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:47:37.137539 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:47:37.137443    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:48:05.860629 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:05.860597    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"]
Jul 21 14:48:05.861082 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:05.860850    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx" podUID="39b22f77-f0a8-494a-9cac-45076725869d" containerName="maas-api" containerID="cri-o://8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437" gracePeriod=30
Jul 21 14:48:06.097223 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.097201    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:48:06.120505 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.120442    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b22f77-f0a8-494a-9cac-45076725869d-openshift-service-ca\") pod \"39b22f77-f0a8-494a-9cac-45076725869d\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") "
Jul 21 14:48:06.120505 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.120485    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7thx\" (UniqueName: \"kubernetes.io/projected/39b22f77-f0a8-494a-9cac-45076725869d-kube-api-access-q7thx\") pod \"39b22f77-f0a8-494a-9cac-45076725869d\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") "
Jul 21 14:48:06.120687 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.120534    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39b22f77-f0a8-494a-9cac-45076725869d-maas-api-tls\") pod \"39b22f77-f0a8-494a-9cac-45076725869d\" (UID: \"39b22f77-f0a8-494a-9cac-45076725869d\") "
Jul 21 14:48:06.120766 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.120740    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b22f77-f0a8-494a-9cac-45076725869d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39b22f77-f0a8-494a-9cac-45076725869d" (UID: "39b22f77-f0a8-494a-9cac-45076725869d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:48:06.122699 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.122674    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b22f77-f0a8-494a-9cac-45076725869d-kube-api-access-q7thx" (OuterVolumeSpecName: "kube-api-access-q7thx") pod "39b22f77-f0a8-494a-9cac-45076725869d" (UID: "39b22f77-f0a8-494a-9cac-45076725869d"). InnerVolumeSpecName "kube-api-access-q7thx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:48:06.122808 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.122770    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b22f77-f0a8-494a-9cac-45076725869d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39b22f77-f0a8-494a-9cac-45076725869d" (UID: "39b22f77-f0a8-494a-9cac-45076725869d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:48:06.222165 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.222125    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b22f77-f0a8-494a-9cac-45076725869d-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:48:06.222165 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.222155    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7thx\" (UniqueName: \"kubernetes.io/projected/39b22f77-f0a8-494a-9cac-45076725869d-kube-api-access-q7thx\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:48:06.222165 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.222165    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39b22f77-f0a8-494a-9cac-45076725869d-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:48:06.959128 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.959097    2569 generic.go:356] "Generic (PLEG): container finished" podID="39b22f77-f0a8-494a-9cac-45076725869d" containerID="8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437" exitCode=0
Jul 21 14:48:06.959628 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.959143    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"
Jul 21 14:48:06.959628 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.959192    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx" event={"ID":"39b22f77-f0a8-494a-9cac-45076725869d","Type":"ContainerDied","Data":"8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437"}
Jul 21 14:48:06.959628 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.959243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx" event={"ID":"39b22f77-f0a8-494a-9cac-45076725869d","Type":"ContainerDied","Data":"ec5c159a049e7988ffcb7f6c4008081de505652b647964bbe4a88a7f0343e09f"}
Jul 21 14:48:06.959628 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.959264    2569 scope.go:117] "RemoveContainer" containerID="8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437"
Jul 21 14:48:06.967725 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.967706    2569 scope.go:117] "RemoveContainer" containerID="8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437"
Jul 21 14:48:06.967972 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:48:06.967953    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437\": container with ID starting with 8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437 not found: ID does not exist" containerID="8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437"
Jul 21 14:48:06.968041 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.967981    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437"} err="failed to get container status \"8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437\": rpc error: code = NotFound desc = could not find container \"8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437\": container with ID starting with 8a1f0d86828741cbe10501210f694572643f3486f469d235cc08ce4eca0ba437 not found: ID does not exist"
Jul 21 14:48:06.992628 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.992593    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"]
Jul 21 14:48:06.996390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:06.996367    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a74b282-6b4ff784d6-b4cwx"]
Jul 21 14:48:08.428380 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:08.428346    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:48:08.468922 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:08.468889    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b22f77-f0a8-494a-9cac-45076725869d" path="/var/lib/kubelet/pods/39b22f77-f0a8-494a-9cac-45076725869d/volumes"
Jul 21 14:48:14.739087 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.739057    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"]
Jul 21 14:48:14.739594 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.739358    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b22f77-f0a8-494a-9cac-45076725869d" containerName="maas-api"
Jul 21 14:48:14.739594 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.739368    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b22f77-f0a8-494a-9cac-45076725869d" containerName="maas-api"
Jul 21 14:48:14.739594 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.739436    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b22f77-f0a8-494a-9cac-45076725869d" containerName="maas-api"
Jul 21 14:48:14.742260 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.742240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.744459 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.744435    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-e24b8b08-serving-cert\""
Jul 21 14:48:14.744594 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.744476    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:48:14.745053 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.745028    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:48:14.745189 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.745107    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:48:14.760903 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.760879    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"]
Jul 21 14:48:14.796676 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.796646    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-openshift-service-ca\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.796857 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.796778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-maas-api-tls\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.796857 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.796821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8h7q\" (UniqueName: \"kubernetes.io/projected/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-kube-api-access-v8h7q\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.898085 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.898048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-maas-api-tls\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.898085 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.898089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8h7q\" (UniqueName: \"kubernetes.io/projected/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-kube-api-access-v8h7q\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.898325 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.898129    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-openshift-service-ca\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.898794 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.898774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-openshift-service-ca\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.900676 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.900654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-maas-api-tls\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:14.907199 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:14.907174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8h7q\" (UniqueName: \"kubernetes.io/projected/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-kube-api-access-v8h7q\") pod \"maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:15.052273 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:15.052200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:15.190366 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:15.190346    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"]
Jul 21 14:48:15.193017 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:48:15.192990    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27460e7_06f9_490e_8ce9_f6fde8b6b9e6.slice/crio-141b517e1dbf3a9983658319fe9d0c548feefc197fff7449a592391396a4e94b WatchSource:0}: Error finding container 141b517e1dbf3a9983658319fe9d0c548feefc197fff7449a592391396a4e94b: Status 404 returned error can't find the container with id 141b517e1dbf3a9983658319fe9d0c548feefc197fff7449a592391396a4e94b
Jul 21 14:48:15.999140 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:15.999113    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx" event={"ID":"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6","Type":"ContainerStarted","Data":"141b517e1dbf3a9983658319fe9d0c548feefc197fff7449a592391396a4e94b"}
Jul 21 14:48:17.005537 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:17.005501    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx" event={"ID":"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6","Type":"ContainerStarted","Data":"4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d"}
Jul 21 14:48:17.005908 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:17.005600    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:17.032034 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:17.031987    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx" podStartSLOduration=2.274931564 podStartE2EDuration="3.031971369s" totalImagesPullingTime="757.039805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:48:17.030750042 +0000 UTC m=+1823.165145407" watchObservedRunningTime="2026-07-21 14:48:17.031971369 +0000 UTC m=+1823.166366743"
Jul 21 14:48:20.927154 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:20.927113    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:48:23.015350 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:23.015317    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:50.114555 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.114520    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"]
Jul 21 14:48:50.115140 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.114734    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx" podUID="e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" containerName="maas-api" containerID="cri-o://4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d" gracePeriod=30
Jul 21 14:48:50.354610 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.354585    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:50.411623 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.411543    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-openshift-service-ca\") pod \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") "
Jul 21 14:48:50.411623 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.411613    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8h7q\" (UniqueName: \"kubernetes.io/projected/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-kube-api-access-v8h7q\") pod \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") "
Jul 21 14:48:50.411811 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.411669    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-maas-api-tls\") pod \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\" (UID: \"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6\") "
Jul 21 14:48:50.411913 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.411890    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" (UID: "e27460e7-06f9-490e-8ce9-f6fde8b6b9e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:48:50.413851 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.413822    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" (UID: "e27460e7-06f9-490e-8ce9-f6fde8b6b9e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:48:50.413966 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.413947    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-kube-api-access-v8h7q" (OuterVolumeSpecName: "kube-api-access-v8h7q") pod "e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" (UID: "e27460e7-06f9-490e-8ce9-f6fde8b6b9e6"). InnerVolumeSpecName "kube-api-access-v8h7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:48:50.512826 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.512795    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:48:50.512826 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.512820    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:48:50.512826 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:50.512831    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8h7q\" (UniqueName: \"kubernetes.io/projected/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6-kube-api-access-v8h7q\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:48:51.146215 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.146185    2569 generic.go:356] "Generic (PLEG): container finished" podID="e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" containerID="4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d" exitCode=0
Jul 21 14:48:51.146627 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.146230    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"
Jul 21 14:48:51.146627 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.146254    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx" event={"ID":"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6","Type":"ContainerDied","Data":"4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d"}
Jul 21 14:48:51.146627 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.146293    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx" event={"ID":"e27460e7-06f9-490e-8ce9-f6fde8b6b9e6","Type":"ContainerDied","Data":"141b517e1dbf3a9983658319fe9d0c548feefc197fff7449a592391396a4e94b"}
Jul 21 14:48:51.146627 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.146316    2569 scope.go:117] "RemoveContainer" containerID="4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d"
Jul 21 14:48:51.154755 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.154735    2569 scope.go:117] "RemoveContainer" containerID="4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d"
Jul 21 14:48:51.155033 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:48:51.155002    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d\": container with ID starting with 4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d not found: ID does not exist" containerID="4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d"
Jul 21 14:48:51.155120 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.155043    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d"} err="failed to get container status \"4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d\": rpc error: code = NotFound desc = could not find container \"4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d\": container with ID starting with 4a31fdd89bb88035f5a3c190b714a7e99fa2ccf8db0d65a11113d780617e967d not found: ID does not exist"
Jul 21 14:48:51.166944 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.166918    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"]
Jul 21 14:48:51.172330 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:51.172310    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e24b8b08-5c9bb97c4-qdgqx"]
Jul 21 14:48:52.467519 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:52.467486    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" path="/var/lib/kubelet/pods/e27460e7-06f9-490e-8ce9-f6fde8b6b9e6/volumes"
Jul 21 14:48:52.926481 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:48:52.926377    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:50:28.339703 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:28.339669    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:50:37.247385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:37.247350    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:50:41.125394 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.125362    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"]
Jul 21 14:50:41.125864 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.125687    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" containerName="maas-api"
Jul 21 14:50:41.125864 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.125698    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" containerName="maas-api"
Jul 21 14:50:41.125864 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.125745    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e27460e7-06f9-490e-8ce9-f6fde8b6b9e6" containerName="maas-api"
Jul 21 14:50:41.128708 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.128687    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.130771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.130720    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:50:41.130771 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.130752    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:50:41.130945 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.130785    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-15f01a6b-serving-cert\""
Jul 21 14:50:41.131451 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.131434    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:50:41.153461 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.153402    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"]
Jul 21 14:50:41.200326 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.200298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.200511 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.200336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c499114c-b176-4f44-a60b-70510e32a30d-openshift-service-ca\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.200511 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.200405    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8khq4\" (UniqueName: \"kubernetes.io/projected/c499114c-b176-4f44-a60b-70510e32a30d-kube-api-access-8khq4\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.301566 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.301530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.301759 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.301573    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c499114c-b176-4f44-a60b-70510e32a30d-openshift-service-ca\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.301759 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.301631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khq4\" (UniqueName: \"kubernetes.io/projected/c499114c-b176-4f44-a60b-70510e32a30d-kube-api-access-8khq4\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.301759 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:50:41.301687    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-serving-cert: secret "maas-api-e2e-mt-15f01a6b-serving-cert" not found
Jul 21 14:50:41.301920 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:50:41.301766    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls podName:c499114c-b176-4f44-a60b-70510e32a30d nodeName:}" failed. No retries permitted until 2026-07-21 14:50:41.801747727 +0000 UTC m=+1967.936143082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls") pod "maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" (UID: "c499114c-b176-4f44-a60b-70510e32a30d") : secret "maas-api-e2e-mt-15f01a6b-serving-cert" not found
Jul 21 14:50:41.302237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.302218    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c499114c-b176-4f44-a60b-70510e32a30d-openshift-service-ca\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.313226 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.313197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8khq4\" (UniqueName: \"kubernetes.io/projected/c499114c-b176-4f44-a60b-70510e32a30d-kube-api-access-8khq4\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.805613 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.805583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:41.808126 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:41.808107    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls\") pod \"maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:42.038330 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:42.038293    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:42.175151 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:42.175117    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"]
Jul 21 14:50:42.178248 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:50:42.178221    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc499114c_b176_4f44_a60b_70510e32a30d.slice/crio-c7de662f7aa436d8668c76cef6dcf3f48702b589428388aba2f64ef85dbee2b1 WatchSource:0}: Error finding container c7de662f7aa436d8668c76cef6dcf3f48702b589428388aba2f64ef85dbee2b1: Status 404 returned error can't find the container with id c7de662f7aa436d8668c76cef6dcf3f48702b589428388aba2f64ef85dbee2b1
Jul 21 14:50:42.620823 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:42.620787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" event={"ID":"c499114c-b176-4f44-a60b-70510e32a30d","Type":"ContainerStarted","Data":"c7de662f7aa436d8668c76cef6dcf3f48702b589428388aba2f64ef85dbee2b1"}
Jul 21 14:50:43.626308 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:43.626276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" event={"ID":"c499114c-b176-4f44-a60b-70510e32a30d","Type":"ContainerStarted","Data":"8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4"}
Jul 21 14:50:43.626789 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:43.626386    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:50:43.646856 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:43.646801    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" podStartSLOduration=2.109313458 podStartE2EDuration="2.646782691s" totalImagesPullingTime="537.469233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:50:43.645353288 +0000 UTC m=+1969.779748676" watchObservedRunningTime="2026-07-21 14:50:43.646782691 +0000 UTC m=+1969.781178064"
Jul 21 14:50:46.820514 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:46.820431    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:50:49.636208 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:50:49.636180    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:51:04.393034 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.393003    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"]
Jul 21 14:51:04.393550 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.393211    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" podUID="c499114c-b176-4f44-a60b-70510e32a30d" containerName="maas-api" containerID="cri-o://8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4" gracePeriod=30
Jul 21 14:51:04.638150 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.638126    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:51:04.711646 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.711619    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls\") pod \"c499114c-b176-4f44-a60b-70510e32a30d\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") "
Jul 21 14:51:04.711809 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.711710    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8khq4\" (UniqueName: \"kubernetes.io/projected/c499114c-b176-4f44-a60b-70510e32a30d-kube-api-access-8khq4\") pod \"c499114c-b176-4f44-a60b-70510e32a30d\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") "
Jul 21 14:51:04.711809 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.711753    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c499114c-b176-4f44-a60b-70510e32a30d-openshift-service-ca\") pod \"c499114c-b176-4f44-a60b-70510e32a30d\" (UID: \"c499114c-b176-4f44-a60b-70510e32a30d\") "
Jul 21 14:51:04.712180 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.712156    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c499114c-b176-4f44-a60b-70510e32a30d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c499114c-b176-4f44-a60b-70510e32a30d" (UID: "c499114c-b176-4f44-a60b-70510e32a30d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:51:04.714025 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.713996    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c499114c-b176-4f44-a60b-70510e32a30d-kube-api-access-8khq4" (OuterVolumeSpecName: "kube-api-access-8khq4") pod "c499114c-b176-4f44-a60b-70510e32a30d" (UID: "c499114c-b176-4f44-a60b-70510e32a30d"). InnerVolumeSpecName "kube-api-access-8khq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:51:04.714152 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.714030    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c499114c-b176-4f44-a60b-70510e32a30d" (UID: "c499114c-b176-4f44-a60b-70510e32a30d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:51:04.714976 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.714949    2569 generic.go:356] "Generic (PLEG): container finished" podID="c499114c-b176-4f44-a60b-70510e32a30d" containerID="8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4" exitCode=0
Jul 21 14:51:04.715068 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.714993    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"
Jul 21 14:51:04.715068 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.715010    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" event={"ID":"c499114c-b176-4f44-a60b-70510e32a30d","Type":"ContainerDied","Data":"8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4"}
Jul 21 14:51:04.715151 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.715074    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" event={"ID":"c499114c-b176-4f44-a60b-70510e32a30d","Type":"ContainerDied","Data":"c7de662f7aa436d8668c76cef6dcf3f48702b589428388aba2f64ef85dbee2b1"}
Jul 21 14:51:04.715151 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.715102    2569 scope.go:117] "RemoveContainer" containerID="8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4"
Jul 21 14:51:04.729992 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.729973    2569 scope.go:117] "RemoveContainer" containerID="8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4"
Jul 21 14:51:04.730240 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:51:04.730222    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4\": container with ID starting with 8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4 not found: ID does not exist" containerID="8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4"
Jul 21 14:51:04.730286 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.730249    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4"} err="failed to get container status \"8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4\": rpc error: code = NotFound desc = could not find container \"8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4\": container with ID starting with 8a5c8788883974692231e42a77c8494e8077e5756f3def7bf8a4018ef7d338d4 not found: ID does not exist"
Jul 21 14:51:04.747456 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.744823    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"]
Jul 21 14:51:04.749914 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.749884    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck"]
Jul 21 14:51:04.813028 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.813002    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8khq4\" (UniqueName: \"kubernetes.io/projected/c499114c-b176-4f44-a60b-70510e32a30d-kube-api-access-8khq4\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:51:04.813028 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.813024    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c499114c-b176-4f44-a60b-70510e32a30d-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:51:04.813028 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:04.813036    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c499114c-b176-4f44-a60b-70510e32a30d-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:51:06.467962 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:06.467928    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c499114c-b176-4f44-a60b-70510e32a30d" path="/var/lib/kubelet/pods/c499114c-b176-4f44-a60b-70510e32a30d/volumes"
Jul 21 14:51:07.068715 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:07.068680    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:51:07.631988 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:07.631936    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15f01a6b-57876cfbcc-j4pck" podUID="c499114c-b176-4f44-a60b-70510e32a30d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.36:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 21 14:51:39.331150 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.331114    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"]
Jul 21 14:51:39.331626 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.331446    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c499114c-b176-4f44-a60b-70510e32a30d" containerName="maas-api"
Jul 21 14:51:39.331626 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.331458    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c499114c-b176-4f44-a60b-70510e32a30d" containerName="maas-api"
Jul 21 14:51:39.331626 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.331535    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c499114c-b176-4f44-a60b-70510e32a30d" containerName="maas-api"
Jul 21 14:51:39.334534 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.334512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.336685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.336657    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:51:39.336685 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.336678    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:51:39.337174 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.337160    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-02afa090-serving-cert\""
Jul 21 14:51:39.337263 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.337162    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:51:39.346743 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.346721    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"]
Jul 21 14:51:39.411898 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.411866    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-426gt\" (UniqueName: \"kubernetes.io/projected/880a0bb3-9050-4571-8761-e36c5b1f893d-kube-api-access-426gt\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.412059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.411904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/880a0bb3-9050-4571-8761-e36c5b1f893d-openshift-service-ca\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.412059 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.411988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.512844 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.512809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-426gt\" (UniqueName: \"kubernetes.io/projected/880a0bb3-9050-4571-8761-e36c5b1f893d-kube-api-access-426gt\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.512844 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.512851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/880a0bb3-9050-4571-8761-e36c5b1f893d-openshift-service-ca\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.513116 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.512990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.513159 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:51:39.513120    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-serving-cert: secret "maas-api-e2e-mt-02afa090-serving-cert" not found
Jul 21 14:51:39.513220 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:51:39.513206    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls podName:880a0bb3-9050-4571-8761-e36c5b1f893d nodeName:}" failed. No retries permitted until 2026-07-21 14:51:40.013181614 +0000 UTC m=+2026.147576966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls") pod "maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" (UID: "880a0bb3-9050-4571-8761-e36c5b1f893d") : secret "maas-api-e2e-mt-02afa090-serving-cert" not found
Jul 21 14:51:39.513459 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.513412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/880a0bb3-9050-4571-8761-e36c5b1f893d-openshift-service-ca\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:39.522498 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:39.522478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-426gt\" (UniqueName: \"kubernetes.io/projected/880a0bb3-9050-4571-8761-e36c5b1f893d-kube-api-access-426gt\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:40.017719 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:40.017679    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:40.020400 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:40.020376    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls\") pod \"maas-api-e2e-mt-02afa090-594bfd5486-kg5qc\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:40.247474 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:40.247435    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:40.383048 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:40.382976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"]
Jul 21 14:51:40.385273 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:51:40.385237    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod880a0bb3_9050_4571_8761_e36c5b1f893d.slice/crio-3d8e8326e7d1f425ceabafe93d6c840b26508bded171e2bf1552a70faab7102d WatchSource:0}: Error finding container 3d8e8326e7d1f425ceabafe93d6c840b26508bded171e2bf1552a70faab7102d: Status 404 returned error can't find the container with id 3d8e8326e7d1f425ceabafe93d6c840b26508bded171e2bf1552a70faab7102d
Jul 21 14:51:40.386583 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:40.386560    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:51:40.869250 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:40.869222    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" event={"ID":"880a0bb3-9050-4571-8761-e36c5b1f893d","Type":"ContainerStarted","Data":"3d8e8326e7d1f425ceabafe93d6c840b26508bded171e2bf1552a70faab7102d"}
Jul 21 14:51:41.875487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:41.875450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" event={"ID":"880a0bb3-9050-4571-8761-e36c5b1f893d","Type":"ContainerStarted","Data":"c0f0af59fc40ad96e43ac4330b6336308243ad996b6c57fa012528e2bcfc3599"}
Jul 21 14:51:41.875860 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:41.875527    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:51:41.895095 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:41.895047    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" podStartSLOduration=2.463122723 podStartE2EDuration="2.895030916s" totalImagesPullingTime="431.908193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:51:41.893703617 +0000 UTC m=+2028.028098992" watchObservedRunningTime="2026-07-21 14:51:41.895030916 +0000 UTC m=+2028.029426267"
Jul 21 14:51:43.829806 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:43.829769    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:51:47.885372 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:51:47.885338    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:52:04.664335 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.664302    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"]
Jul 21 14:52:04.667508 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.667490    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.670195 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.670172    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-02afa090\"/\"e2e-lifecycle-model-02afa090-kserve-self-signed-certs\""
Jul 21 14:52:04.670319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.670214    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-02afa090\"/\"openshift-service-ca.crt\""
Jul 21 14:52:04.670319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.670173    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-02afa090\"/\"default-dockercfg-qmd4t\""
Jul 21 14:52:04.670319 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.670174    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-02afa090\"/\"kube-root-ca.crt\""
Jul 21 14:52:04.676786 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.676767    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"]
Jul 21 14:52:04.731616 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.731585    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-home\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.731616 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.731624    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b1e8db-471d-4d39-809e-655ff367eff2-tls-certs\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.731849 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.731724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kbbv\" (UniqueName: \"kubernetes.io/projected/c6b1e8db-471d-4d39-809e-655ff367eff2-kube-api-access-5kbbv\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.731849 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.731758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-model-cache\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.731849 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.731798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-tmp-dir\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.731849 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.731820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-dshm\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.832575 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.832533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kbbv\" (UniqueName: \"kubernetes.io/projected/c6b1e8db-471d-4d39-809e-655ff367eff2-kube-api-access-5kbbv\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.832575 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.832581    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-model-cache\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.832841 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.832627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-tmp-dir\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.832841 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.832652    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-dshm\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.832841 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.832710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-home\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.832841 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.832733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b1e8db-471d-4d39-809e-655ff367eff2-tls-certs\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.833079 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.833015    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-tmp-dir\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.833135 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.833084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-model-cache\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.833135 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.833115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-home\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.835112 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.835090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-dshm\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.835374 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.835354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b1e8db-471d-4d39-809e-655ff367eff2-tls-certs\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.841963 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.841934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kbbv\" (UniqueName: \"kubernetes.io/projected/c6b1e8db-471d-4d39-809e-655ff367eff2-kube-api-access-5kbbv\") pod \"e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") " pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:04.977514 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:04.977483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:05.118005 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:05.117968    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"]
Jul 21 14:52:05.121174 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:52:05.121151    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6b1e8db_471d_4d39_809e_655ff367eff2.slice/crio-160a392dfdb8d4a47e178c123b783d3ceaf8aa1294407d07191f9da78bd5d28d WatchSource:0}: Error finding container 160a392dfdb8d4a47e178c123b783d3ceaf8aa1294407d07191f9da78bd5d28d: Status 404 returned error can't find the container with id 160a392dfdb8d4a47e178c123b783d3ceaf8aa1294407d07191f9da78bd5d28d
Jul 21 14:52:05.994527 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:05.994490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v" event={"ID":"c6b1e8db-471d-4d39-809e-655ff367eff2","Type":"ContainerStarted","Data":"356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32"}
Jul 21 14:52:05.995033 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:05.994540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v" event={"ID":"c6b1e8db-471d-4d39-809e-655ff367eff2","Type":"ContainerStarted","Data":"160a392dfdb8d4a47e178c123b783d3ceaf8aa1294407d07191f9da78bd5d28d"}
Jul 21 14:52:06.014690 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:06.014640    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v" podStartSLOduration=2.014625185 podStartE2EDuration="2.014625185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:52:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:52:06.013276009 +0000 UTC m=+2052.147671385" watchObservedRunningTime="2026-07-21 14:52:06.014625185 +0000 UTC m=+2052.149020559"
Jul 21 14:52:07.718942 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:07.718907    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:52:14.978294 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:14.978207    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:14.978294 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:14.978278    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:14.990956 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:14.990936    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:15.041832 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:15.041809    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:52:18.934050 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:18.934016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:52:36.321678 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:36.321641    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:52:37.908052 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:37.908020    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"]
Jul 21 14:52:37.908456 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:37.908210    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" podUID="880a0bb3-9050-4571-8761-e36c5b1f893d" containerName="maas-api" containerID="cri-o://c0f0af59fc40ad96e43ac4330b6336308243ad996b6c57fa012528e2bcfc3599" gracePeriod=30
Jul 21 14:52:38.127824 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.127790    2569 generic.go:356] "Generic (PLEG): container finished" podID="880a0bb3-9050-4571-8761-e36c5b1f893d" containerID="c0f0af59fc40ad96e43ac4330b6336308243ad996b6c57fa012528e2bcfc3599" exitCode=0
Jul 21 14:52:38.127974 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.127846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" event={"ID":"880a0bb3-9050-4571-8761-e36c5b1f893d","Type":"ContainerDied","Data":"c0f0af59fc40ad96e43ac4330b6336308243ad996b6c57fa012528e2bcfc3599"}
Jul 21 14:52:38.154441 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.154401    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:52:38.335860 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.335834    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls\") pod \"880a0bb3-9050-4571-8761-e36c5b1f893d\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") "
Jul 21 14:52:38.336077 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.335904    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/880a0bb3-9050-4571-8761-e36c5b1f893d-openshift-service-ca\") pod \"880a0bb3-9050-4571-8761-e36c5b1f893d\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") "
Jul 21 14:52:38.336077 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.335937    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-426gt\" (UniqueName: \"kubernetes.io/projected/880a0bb3-9050-4571-8761-e36c5b1f893d-kube-api-access-426gt\") pod \"880a0bb3-9050-4571-8761-e36c5b1f893d\" (UID: \"880a0bb3-9050-4571-8761-e36c5b1f893d\") "
Jul 21 14:52:38.336325 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.336296    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880a0bb3-9050-4571-8761-e36c5b1f893d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "880a0bb3-9050-4571-8761-e36c5b1f893d" (UID: "880a0bb3-9050-4571-8761-e36c5b1f893d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:52:38.338305 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.338274    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880a0bb3-9050-4571-8761-e36c5b1f893d-kube-api-access-426gt" (OuterVolumeSpecName: "kube-api-access-426gt") pod "880a0bb3-9050-4571-8761-e36c5b1f893d" (UID: "880a0bb3-9050-4571-8761-e36c5b1f893d"). InnerVolumeSpecName "kube-api-access-426gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:52:38.338402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.338302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "880a0bb3-9050-4571-8761-e36c5b1f893d" (UID: "880a0bb3-9050-4571-8761-e36c5b1f893d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:52:38.436971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.436937    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/880a0bb3-9050-4571-8761-e36c5b1f893d-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:52:38.436971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.436964    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/880a0bb3-9050-4571-8761-e36c5b1f893d-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:52:38.436971 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:38.436973    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-426gt\" (UniqueName: \"kubernetes.io/projected/880a0bb3-9050-4571-8761-e36c5b1f893d-kube-api-access-426gt\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:52:39.133325 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:39.133293    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"
Jul 21 14:52:39.133325 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:39.133309    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc" event={"ID":"880a0bb3-9050-4571-8761-e36c5b1f893d","Type":"ContainerDied","Data":"3d8e8326e7d1f425ceabafe93d6c840b26508bded171e2bf1552a70faab7102d"}
Jul 21 14:52:39.133799 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:39.133386    2569 scope.go:117] "RemoveContainer" containerID="c0f0af59fc40ad96e43ac4330b6336308243ad996b6c57fa012528e2bcfc3599"
Jul 21 14:52:39.150514 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:39.150480    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"]
Jul 21 14:52:39.157584 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:39.157552    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02afa090-594bfd5486-kg5qc"]
Jul 21 14:52:40.468295 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:40.468260    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="880a0bb3-9050-4571-8761-e36c5b1f893d" path="/var/lib/kubelet/pods/880a0bb3-9050-4571-8761-e36c5b1f893d/volumes"
Jul 21 14:52:46.015927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:46.015893    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:52:49.469364 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:52:49.469325    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"]
Jul 21 14:53:04.489876 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.489814    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v" podUID="c6b1e8db-471d-4d39-809e-655ff367eff2" containerName="main" containerID="cri-o://356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32" gracePeriod=45
Jul 21 14:53:04.761393 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.761365    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:53:04.859706 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859664    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kbbv\" (UniqueName: \"kubernetes.io/projected/c6b1e8db-471d-4d39-809e-655ff367eff2-kube-api-access-5kbbv\") pod \"c6b1e8db-471d-4d39-809e-655ff367eff2\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") "
Jul 21 14:53:04.859706 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859703    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-model-cache\") pod \"c6b1e8db-471d-4d39-809e-655ff367eff2\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") "
Jul 21 14:53:04.859927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-home\") pod \"c6b1e8db-471d-4d39-809e-655ff367eff2\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") "
Jul 21 14:53:04.859927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859756    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-tmp-dir\") pod \"c6b1e8db-471d-4d39-809e-655ff367eff2\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") "
Jul 21 14:53:04.859927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859802    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b1e8db-471d-4d39-809e-655ff367eff2-tls-certs\") pod \"c6b1e8db-471d-4d39-809e-655ff367eff2\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") "
Jul 21 14:53:04.859927 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859897    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-dshm\") pod \"c6b1e8db-471d-4d39-809e-655ff367eff2\" (UID: \"c6b1e8db-471d-4d39-809e-655ff367eff2\") "
Jul 21 14:53:04.860141 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.859987    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6b1e8db-471d-4d39-809e-655ff367eff2" (UID: "c6b1e8db-471d-4d39-809e-655ff367eff2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:53:04.860141 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.860006    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-home" (OuterVolumeSpecName: "home") pod "c6b1e8db-471d-4d39-809e-655ff367eff2" (UID: "c6b1e8db-471d-4d39-809e-655ff367eff2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:53:04.860141 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.860016    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6b1e8db-471d-4d39-809e-655ff367eff2" (UID: "c6b1e8db-471d-4d39-809e-655ff367eff2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:53:04.860275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.860178    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-model-cache\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:53:04.860275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.860197    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-home\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:53:04.860275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.860209    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-tmp-dir\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:53:04.862093 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.862059    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6b1e8db-471d-4d39-809e-655ff367eff2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6b1e8db-471d-4d39-809e-655ff367eff2" (UID: "c6b1e8db-471d-4d39-809e-655ff367eff2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:53:04.862208 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.862106    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6b1e8db-471d-4d39-809e-655ff367eff2-kube-api-access-5kbbv" (OuterVolumeSpecName: "kube-api-access-5kbbv") pod "c6b1e8db-471d-4d39-809e-655ff367eff2" (UID: "c6b1e8db-471d-4d39-809e-655ff367eff2"). InnerVolumeSpecName "kube-api-access-5kbbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:53:04.862208 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.862150    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-dshm" (OuterVolumeSpecName: "dshm") pod "c6b1e8db-471d-4d39-809e-655ff367eff2" (UID: "c6b1e8db-471d-4d39-809e-655ff367eff2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:53:04.961275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.961244    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kbbv\" (UniqueName: \"kubernetes.io/projected/c6b1e8db-471d-4d39-809e-655ff367eff2-kube-api-access-5kbbv\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:53:04.961275 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.961273    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b1e8db-471d-4d39-809e-655ff367eff2-tls-certs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:53:04.961512 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:04.961285    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b1e8db-471d-4d39-809e-655ff367eff2-dshm\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:53:05.243527 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.243491    2569 generic.go:356] "Generic (PLEG): container finished" podID="c6b1e8db-471d-4d39-809e-655ff367eff2" containerID="356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32" exitCode=0
Jul 21 14:53:05.243712 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.243553    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"
Jul 21 14:53:05.243712 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.243580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v" event={"ID":"c6b1e8db-471d-4d39-809e-655ff367eff2","Type":"ContainerDied","Data":"356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32"}
Jul 21 14:53:05.243712 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.243627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v" event={"ID":"c6b1e8db-471d-4d39-809e-655ff367eff2","Type":"ContainerDied","Data":"160a392dfdb8d4a47e178c123b783d3ceaf8aa1294407d07191f9da78bd5d28d"}
Jul 21 14:53:05.243712 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.243646    2569 scope.go:117] "RemoveContainer" containerID="356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32"
Jul 21 14:53:05.252503 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.252485    2569 scope.go:117] "RemoveContainer" containerID="356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32"
Jul 21 14:53:05.252772 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:53:05.252751    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32\": container with ID starting with 356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32 not found: ID does not exist" containerID="356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32"
Jul 21 14:53:05.252865 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.252779    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32"} err="failed to get container status \"356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32\": rpc error: code = NotFound desc = could not find container \"356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32\": container with ID starting with 356ca186751d38e3eb353df4132e600f80b27f97325226b4e417609ad87d3a32 not found: ID does not exist"
Jul 21 14:53:05.267316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.267294    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"]
Jul 21 14:53:05.272228 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:05.272206    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-02afa090/e2e-lifecycle-model-02afa090-kserve-85f67476f5-hwr5v"]
Jul 21 14:53:06.468308 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:06.468278    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6b1e8db-471d-4d39-809e-655ff367eff2" path="/var/lib/kubelet/pods/c6b1e8db-471d-4d39-809e-655ff367eff2/volumes"
Jul 21 14:53:38.338378 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:38.338347    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:53:48.222244 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:53:48.222200    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:54:35.336706 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.336673    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"]
Jul 21 14:54:35.337402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.336986    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880a0bb3-9050-4571-8761-e36c5b1f893d" containerName="maas-api"
Jul 21 14:54:35.337402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.336998    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="880a0bb3-9050-4571-8761-e36c5b1f893d" containerName="maas-api"
Jul 21 14:54:35.337402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.337018    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6b1e8db-471d-4d39-809e-655ff367eff2" containerName="main"
Jul 21 14:54:35.337402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.337024    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6b1e8db-471d-4d39-809e-655ff367eff2" containerName="main"
Jul 21 14:54:35.337402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.337077    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6b1e8db-471d-4d39-809e-655ff367eff2" containerName="main"
Jul 21 14:54:35.337402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.337092    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="880a0bb3-9050-4571-8761-e36c5b1f893d" containerName="maas-api"
Jul 21 14:54:35.339749 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.339729    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.344241 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.344221    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:54:35.344374 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.344254    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-db7687-serving-cert\""
Jul 21 14:54:35.344544 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.344528    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:54:35.344672 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.344651    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:54:35.359353 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.359327    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"]
Jul 21 14:54:35.434524 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.434487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d5867-924e-4cf2-9e1d-868f163f717c-maas-api-tls\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.434524 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.434520    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d5867-924e-4cf2-9e1d-868f163f717c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.434740 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.434550    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5cg2\" (UniqueName: \"kubernetes.io/projected/200d5867-924e-4cf2-9e1d-868f163f717c-kube-api-access-z5cg2\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.535944 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.535895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d5867-924e-4cf2-9e1d-868f163f717c-maas-api-tls\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.536147 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.536067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d5867-924e-4cf2-9e1d-868f163f717c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.536147 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.536115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5cg2\" (UniqueName: \"kubernetes.io/projected/200d5867-924e-4cf2-9e1d-868f163f717c-kube-api-access-z5cg2\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.536653 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.536631    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d5867-924e-4cf2-9e1d-868f163f717c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.538515 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.538490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d5867-924e-4cf2-9e1d-868f163f717c-maas-api-tls\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.545123 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.545102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5cg2\" (UniqueName: \"kubernetes.io/projected/200d5867-924e-4cf2-9e1d-868f163f717c-kube-api-access-z5cg2\") pod \"maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.651555 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.651478    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:35.805809 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:35.805745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"]
Jul 21 14:54:35.808561 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:54:35.808532    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200d5867_924e_4cf2_9e1d_868f163f717c.slice/crio-06ad7b3c4c235044e77652fa4f3c4f5579e9c465a043b5b134bfb65a19423a9d WatchSource:0}: Error finding container 06ad7b3c4c235044e77652fa4f3c4f5579e9c465a043b5b134bfb65a19423a9d: Status 404 returned error can't find the container with id 06ad7b3c4c235044e77652fa4f3c4f5579e9c465a043b5b134bfb65a19423a9d
Jul 21 14:54:36.641200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:36.641113    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh" event={"ID":"200d5867-924e-4cf2-9e1d-868f163f717c","Type":"ContainerStarted","Data":"63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46"}
Jul 21 14:54:36.641200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:36.641177    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh" event={"ID":"200d5867-924e-4cf2-9e1d-868f163f717c","Type":"ContainerStarted","Data":"06ad7b3c4c235044e77652fa4f3c4f5579e9c465a043b5b134bfb65a19423a9d"}
Jul 21 14:54:36.641653 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:36.641252    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:40.028933 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:40.028877    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh" podStartSLOduration=4.57114742 podStartE2EDuration="5.028860846s" totalImagesPullingTime="457.713426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:54:36.678785943 +0000 UTC m=+2202.813181317" watchObservedRunningTime="2026-07-21 14:54:40.028860846 +0000 UTC m=+2206.163256220"
Jul 21 14:54:40.029961 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:40.029942    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:54:42.651146 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:42.651117    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:54:43.465462 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.465411    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"]
Jul 21 14:54:43.469565 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.469543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.471935 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.471913    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-6c5a8f-serving-cert\""
Jul 21 14:54:43.486413 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.486385    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"]
Jul 21 14:54:43.510630 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.510602    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7njg\" (UniqueName: \"kubernetes.io/projected/ac3cd7ac-9512-480c-b59b-7d14418e71d7-kube-api-access-w7njg\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.510748 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.510637    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.510748 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.510662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3cd7ac-9512-480c-b59b-7d14418e71d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.612225 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.612192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7njg\" (UniqueName: \"kubernetes.io/projected/ac3cd7ac-9512-480c-b59b-7d14418e71d7-kube-api-access-w7njg\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.612225 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.612226    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.612450 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.612245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3cd7ac-9512-480c-b59b-7d14418e71d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.612450 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:54:43.612353    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-serving-cert: secret "maas-api-e2e-inf-b-6c5a8f-serving-cert" not found
Jul 21 14:54:43.612450 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:54:43.612439    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls podName:ac3cd7ac-9512-480c-b59b-7d14418e71d7 nodeName:}" failed. No retries permitted until 2026-07-21 14:54:44.11240338 +0000 UTC m=+2210.246798736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls") pod "maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" (UID: "ac3cd7ac-9512-480c-b59b-7d14418e71d7") : secret "maas-api-e2e-inf-b-6c5a8f-serving-cert" not found
Jul 21 14:54:43.612836 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.612820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3cd7ac-9512-480c-b59b-7d14418e71d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:43.622652 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:43.622627    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7njg\" (UniqueName: \"kubernetes.io/projected/ac3cd7ac-9512-480c-b59b-7d14418e71d7-kube-api-access-w7njg\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:44.116959 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:44.116929    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:44.119544 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:44.119518    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:44.380104 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:44.380014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:44.541829 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:44.541789    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"]
Jul 21 14:54:44.542078 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:54:44.542054    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3cd7ac_9512_480c_b59b_7d14418e71d7.slice/crio-cf2a75c66064baa919cc63b5c4acdff895021ee01b74705db702be542596f724 WatchSource:0}: Error finding container cf2a75c66064baa919cc63b5c4acdff895021ee01b74705db702be542596f724: Status 404 returned error can't find the container with id cf2a75c66064baa919cc63b5c4acdff895021ee01b74705db702be542596f724
Jul 21 14:54:44.676750 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:44.676666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" event={"ID":"ac3cd7ac-9512-480c-b59b-7d14418e71d7","Type":"ContainerStarted","Data":"cf2a75c66064baa919cc63b5c4acdff895021ee01b74705db702be542596f724"}
Jul 21 14:54:45.682487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:45.682455    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" event={"ID":"ac3cd7ac-9512-480c-b59b-7d14418e71d7","Type":"ContainerStarted","Data":"ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b"}
Jul 21 14:54:45.682918 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:45.682545    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:54:45.823003 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:45.822953    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" podStartSLOduration=2.368548363 podStartE2EDuration="2.822934067s" totalImagesPullingTime="454.385704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:54:45.819914476 +0000 UTC m=+2211.954309851" watchObservedRunningTime="2026-07-21 14:54:45.822934067 +0000 UTC m=+2211.957329441"
Jul 21 14:54:48.642204 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:48.642167    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:54:51.693051 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:54:51.693024    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:55:17.340410 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:55:17.340321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:55:29.712447 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:55:29.712391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:55:49.621164 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:55:49.621127    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:56:02.414667 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:02.414631    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:56:37.748727 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:37.748695    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:56:44.682372 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:44.682158    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"]
Jul 21 14:56:44.682836 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:44.682442    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh" podUID="200d5867-924e-4cf2-9e1d-868f163f717c" containerName="maas-api" containerID="cri-o://63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46" gracePeriod=30
Jul 21 14:56:44.939035 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:44.938970    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:56:45.087333 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.087301    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d5867-924e-4cf2-9e1d-868f163f717c-maas-api-tls\") pod \"200d5867-924e-4cf2-9e1d-868f163f717c\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") "
Jul 21 14:56:45.087333 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.087335    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5cg2\" (UniqueName: \"kubernetes.io/projected/200d5867-924e-4cf2-9e1d-868f163f717c-kube-api-access-z5cg2\") pod \"200d5867-924e-4cf2-9e1d-868f163f717c\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") "
Jul 21 14:56:45.087606 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.087366    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d5867-924e-4cf2-9e1d-868f163f717c-openshift-service-ca\") pod \"200d5867-924e-4cf2-9e1d-868f163f717c\" (UID: \"200d5867-924e-4cf2-9e1d-868f163f717c\") "
Jul 21 14:56:45.087787 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.087765    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/200d5867-924e-4cf2-9e1d-868f163f717c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "200d5867-924e-4cf2-9e1d-868f163f717c" (UID: "200d5867-924e-4cf2-9e1d-868f163f717c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:56:45.089689 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.089664    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200d5867-924e-4cf2-9e1d-868f163f717c-kube-api-access-z5cg2" (OuterVolumeSpecName: "kube-api-access-z5cg2") pod "200d5867-924e-4cf2-9e1d-868f163f717c" (UID: "200d5867-924e-4cf2-9e1d-868f163f717c"). InnerVolumeSpecName "kube-api-access-z5cg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:56:45.089780 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.089664    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/200d5867-924e-4cf2-9e1d-868f163f717c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "200d5867-924e-4cf2-9e1d-868f163f717c" (UID: "200d5867-924e-4cf2-9e1d-868f163f717c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:56:45.189089 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.188998    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d5867-924e-4cf2-9e1d-868f163f717c-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:56:45.189089 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.189046    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d5867-924e-4cf2-9e1d-868f163f717c-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:56:45.189089 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.189061    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5cg2\" (UniqueName: \"kubernetes.io/projected/200d5867-924e-4cf2-9e1d-868f163f717c-kube-api-access-z5cg2\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:56:45.202730 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.202698    2569 generic.go:356] "Generic (PLEG): container finished" podID="200d5867-924e-4cf2-9e1d-868f163f717c" containerID="63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46" exitCode=0
Jul 21 14:56:45.202866 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.202740    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"
Jul 21 14:56:45.202866 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.202789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh" event={"ID":"200d5867-924e-4cf2-9e1d-868f163f717c","Type":"ContainerDied","Data":"63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46"}
Jul 21 14:56:45.202866 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.202839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh" event={"ID":"200d5867-924e-4cf2-9e1d-868f163f717c","Type":"ContainerDied","Data":"06ad7b3c4c235044e77652fa4f3c4f5579e9c465a043b5b134bfb65a19423a9d"}
Jul 21 14:56:45.202866 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.202860    2569 scope.go:117] "RemoveContainer" containerID="63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46"
Jul 21 14:56:45.211687 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.211669    2569 scope.go:117] "RemoveContainer" containerID="63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46"
Jul 21 14:56:45.211945 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:56:45.211929    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46\": container with ID starting with 63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46 not found: ID does not exist" containerID="63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46"
Jul 21 14:56:45.211992 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.211951    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46"} err="failed to get container status \"63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46\": rpc error: code = NotFound desc = could not find container \"63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46\": container with ID starting with 63abfb81f18b5fe8ab0c812a64c43165253676191641256bbb85809edc286c46 not found: ID does not exist"
Jul 21 14:56:45.226883 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.226861    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"]
Jul 21 14:56:45.231358 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:45.231339    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db7687-5dccdfd67f-6t2bh"]
Jul 21 14:56:46.468331 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:46.468295    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="200d5867-924e-4cf2-9e1d-868f163f717c" path="/var/lib/kubelet/pods/200d5867-924e-4cf2-9e1d-868f163f717c/volumes"
Jul 21 14:56:53.054669 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:56:53.054623    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:57:07.839301 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:07.839268    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"]
Jul 21 14:57:07.839740 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:07.839501    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" podUID="ac3cd7ac-9512-480c-b59b-7d14418e71d7" containerName="maas-api" containerID="cri-o://ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b" gracePeriod=30
Jul 21 14:57:08.084273 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.084250    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:57:08.160316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.160213    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3cd7ac-9512-480c-b59b-7d14418e71d7-openshift-service-ca\") pod \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") "
Jul 21 14:57:08.160316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.160260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7njg\" (UniqueName: \"kubernetes.io/projected/ac3cd7ac-9512-480c-b59b-7d14418e71d7-kube-api-access-w7njg\") pod \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") "
Jul 21 14:57:08.160316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.160288    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls\") pod \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\" (UID: \"ac3cd7ac-9512-480c-b59b-7d14418e71d7\") "
Jul 21 14:57:08.160710 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.160684    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac3cd7ac-9512-480c-b59b-7d14418e71d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac3cd7ac-9512-480c-b59b-7d14418e71d7" (UID: "ac3cd7ac-9512-480c-b59b-7d14418e71d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:57:08.162614 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.162580    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac3cd7ac-9512-480c-b59b-7d14418e71d7-kube-api-access-w7njg" (OuterVolumeSpecName: "kube-api-access-w7njg") pod "ac3cd7ac-9512-480c-b59b-7d14418e71d7" (UID: "ac3cd7ac-9512-480c-b59b-7d14418e71d7"). InnerVolumeSpecName "kube-api-access-w7njg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:57:08.162614 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.162607    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac3cd7ac-9512-480c-b59b-7d14418e71d7" (UID: "ac3cd7ac-9512-480c-b59b-7d14418e71d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:57:08.261347 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.261314    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3cd7ac-9512-480c-b59b-7d14418e71d7-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:57:08.261347 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.261341    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7njg\" (UniqueName: \"kubernetes.io/projected/ac3cd7ac-9512-480c-b59b-7d14418e71d7-kube-api-access-w7njg\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:57:08.261347 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.261352    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3cd7ac-9512-480c-b59b-7d14418e71d7-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:57:08.300969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.299134    2569 generic.go:356] "Generic (PLEG): container finished" podID="ac3cd7ac-9512-480c-b59b-7d14418e71d7" containerID="ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b" exitCode=0
Jul 21 14:57:08.300969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.299257    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" event={"ID":"ac3cd7ac-9512-480c-b59b-7d14418e71d7","Type":"ContainerDied","Data":"ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b"}
Jul 21 14:57:08.300969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.299304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6" event={"ID":"ac3cd7ac-9512-480c-b59b-7d14418e71d7","Type":"ContainerDied","Data":"cf2a75c66064baa919cc63b5c4acdff895021ee01b74705db702be542596f724"}
Jul 21 14:57:08.300969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.299331    2569 scope.go:117] "RemoveContainer" containerID="ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b"
Jul 21 14:57:08.300969 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.299527    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"
Jul 21 14:57:08.310155 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.310137    2569 scope.go:117] "RemoveContainer" containerID="ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b"
Jul 21 14:57:08.310406 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:57:08.310388    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b\": container with ID starting with ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b not found: ID does not exist" containerID="ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b"
Jul 21 14:57:08.310495 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.310432    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b"} err="failed to get container status \"ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b\": rpc error: code = NotFound desc = could not find container \"ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b\": container with ID starting with ab15d271b74633caa4d41550965f83d7cf736a91e75345a8fe7259e68bed342b not found: ID does not exist"
Jul 21 14:57:08.324641 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.324617    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"]
Jul 21 14:57:08.330085 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.330065    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6c5a8f-64667d6bb9-f82d6"]
Jul 21 14:57:08.468440 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:08.468397    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac3cd7ac-9512-480c-b59b-7d14418e71d7" path="/var/lib/kubelet/pods/ac3cd7ac-9512-480c-b59b-7d14418e71d7/volumes"
Jul 21 14:57:10.824255 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:10.824224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:57:23.286481 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286445    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"]
Jul 21 14:57:23.286847 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286763    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac3cd7ac-9512-480c-b59b-7d14418e71d7" containerName="maas-api"
Jul 21 14:57:23.286847 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286774    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac3cd7ac-9512-480c-b59b-7d14418e71d7" containerName="maas-api"
Jul 21 14:57:23.286847 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286791    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200d5867-924e-4cf2-9e1d-868f163f717c" containerName="maas-api"
Jul 21 14:57:23.286847 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286796    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="200d5867-924e-4cf2-9e1d-868f163f717c" containerName="maas-api"
Jul 21 14:57:23.286986 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286858    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="200d5867-924e-4cf2-9e1d-868f163f717c" containerName="maas-api"
Jul 21 14:57:23.286986 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.286865    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac3cd7ac-9512-480c-b59b-7d14418e71d7" containerName="maas-api"
Jul 21 14:57:23.291295 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.291277    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.295082 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.295061    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:57:23.295630 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.295615    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:57:23.295696 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.295627    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:57:23.295696 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.295616    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6ecdf5-serving-cert\""
Jul 21 14:57:23.302966 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.302941    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"]
Jul 21 14:57:23.394600 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.394568    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-maas-api-tls\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.394796 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.394672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqlb4\" (UniqueName: \"kubernetes.io/projected/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-kube-api-access-xqlb4\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.394796 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.394767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-openshift-service-ca\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.496237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.496197    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-openshift-service-ca\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.496437 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.496254    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-maas-api-tls\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.496437 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.496285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqlb4\" (UniqueName: \"kubernetes.io/projected/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-kube-api-access-xqlb4\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.496905 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.496876    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-openshift-service-ca\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.498769 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.498750    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-maas-api-tls\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.505086 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.505064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqlb4\" (UniqueName: \"kubernetes.io/projected/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-kube-api-access-xqlb4\") pod \"maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.601747 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.601668    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:23.739169 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.739133    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"]
Jul 21 14:57:23.742337 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:57:23.742304    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af9e7fd_0cd6_4285_b87c_bcec8fc2ff2e.slice/crio-f8ed3556fa58a0bbb7123fca04b4dd6623ea6fab7c97c2a1ec15f736c33697b8 WatchSource:0}: Error finding container f8ed3556fa58a0bbb7123fca04b4dd6623ea6fab7c97c2a1ec15f736c33697b8: Status 404 returned error can't find the container with id f8ed3556fa58a0bbb7123fca04b4dd6623ea6fab7c97c2a1ec15f736c33697b8
Jul 21 14:57:23.744072 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:23.744054    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:57:24.368937 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:24.368902    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7" event={"ID":"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e","Type":"ContainerStarted","Data":"f8ed3556fa58a0bbb7123fca04b4dd6623ea6fab7c97c2a1ec15f736c33697b8"}
Jul 21 14:57:25.374120 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:25.374087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7" event={"ID":"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e","Type":"ContainerStarted","Data":"46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70"}
Jul 21 14:57:25.374558 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:25.374203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:25.393544 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:25.393501    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7" podStartSLOduration=1.7316303629999998 podStartE2EDuration="2.393487107s" totalImagesPullingTime="661.856744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:57:25.392258794 +0000 UTC m=+2371.526654179" watchObservedRunningTime="2026-07-21 14:57:25.393487107 +0000 UTC m=+2371.527882484"
Jul 21 14:57:26.024718 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:26.024682    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:57:31.359889 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.359854    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"]
Jul 21 14:57:31.363255 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.363231    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.365617 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.365591    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-63bc89-serving-cert\""
Jul 21 14:57:31.376203 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.376163    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"]
Jul 21 14:57:31.387869 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.387849    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:31.464605 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.464571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19ed5989-d23a-4bc7-b42d-b5dd08e93840-maas-api-tls\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.464781 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.464618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ed5989-d23a-4bc7-b42d-b5dd08e93840-openshift-service-ca\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.464781 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.464703    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqtp\" (UniqueName: \"kubernetes.io/projected/19ed5989-d23a-4bc7-b42d-b5dd08e93840-kube-api-access-pfqtp\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.565634 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.565595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ed5989-d23a-4bc7-b42d-b5dd08e93840-openshift-service-ca\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.565804 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.565650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqtp\" (UniqueName: \"kubernetes.io/projected/19ed5989-d23a-4bc7-b42d-b5dd08e93840-kube-api-access-pfqtp\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.565804 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.565782    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19ed5989-d23a-4bc7-b42d-b5dd08e93840-maas-api-tls\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.566396 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.566377    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ed5989-d23a-4bc7-b42d-b5dd08e93840-openshift-service-ca\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.568316 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.568298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19ed5989-d23a-4bc7-b42d-b5dd08e93840-maas-api-tls\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.573963 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.573936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqtp\" (UniqueName: \"kubernetes.io/projected/19ed5989-d23a-4bc7-b42d-b5dd08e93840-kube-api-access-pfqtp\") pod \"maas-api-e2e-api-b-63bc89-858768bf88-m768g\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.678092 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.678008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:31.817657 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:31.817622    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"]
Jul 21 14:57:31.821735 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:57:31.821710    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ed5989_d23a_4bc7_b42d_b5dd08e93840.slice/crio-d787815ee314586061763870687d75a4c7cb3c63f0e83bbf2773e745b788944a WatchSource:0}: Error finding container d787815ee314586061763870687d75a4c7cb3c63f0e83bbf2773e745b788944a: Status 404 returned error can't find the container with id d787815ee314586061763870687d75a4c7cb3c63f0e83bbf2773e745b788944a
Jul 21 14:57:32.409769 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:32.409723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g" event={"ID":"19ed5989-d23a-4bc7-b42d-b5dd08e93840","Type":"ContainerStarted","Data":"da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199"}
Jul 21 14:57:32.410200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:32.409819    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:32.410200 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:32.409868    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g" event={"ID":"19ed5989-d23a-4bc7-b42d-b5dd08e93840","Type":"ContainerStarted","Data":"d787815ee314586061763870687d75a4c7cb3c63f0e83bbf2773e745b788944a"}
Jul 21 14:57:32.436187 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:32.433174    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g" podStartSLOduration=1.084303282 podStartE2EDuration="1.43315072s" totalImagesPullingTime="348.847438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:57:32.429689581 +0000 UTC m=+2378.564084955" watchObservedRunningTime="2026-07-21 14:57:32.43315072 +0000 UTC m=+2378.567546095"
Jul 21 14:57:36.816016 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:36.815978    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:57:38.419978 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:38.419949    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:57:58.846183 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:58.846150    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"]
Jul 21 14:57:58.846688 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:58.846350    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7" podUID="7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" containerName="maas-api" containerID="cri-o://46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70" gracePeriod=30
Jul 21 14:57:59.098761 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.098698    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:59.208238 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.208202    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqlb4\" (UniqueName: \"kubernetes.io/projected/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-kube-api-access-xqlb4\") pod \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") "
Jul 21 14:57:59.208402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.208247    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-openshift-service-ca\") pod \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") "
Jul 21 14:57:59.208402 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.208339    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-maas-api-tls\") pod \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\" (UID: \"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e\") "
Jul 21 14:57:59.208694 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.208663    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" (UID: "7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:57:59.210545 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.210516    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" (UID: "7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:57:59.210656 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.210629    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-kube-api-access-xqlb4" (OuterVolumeSpecName: "kube-api-access-xqlb4") pod "7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" (UID: "7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e"). InnerVolumeSpecName "kube-api-access-xqlb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:57:59.309111 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.309079    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:57:59.309111 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.309105    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqlb4\" (UniqueName: \"kubernetes.io/projected/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-kube-api-access-xqlb4\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:57:59.309111 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.309115    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:57:59.524096 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.524055    2569 generic.go:356] "Generic (PLEG): container finished" podID="7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" containerID="46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70" exitCode=0
Jul 21 14:57:59.524096 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.524100    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"
Jul 21 14:57:59.524300 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.524108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7" event={"ID":"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e","Type":"ContainerDied","Data":"46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70"}
Jul 21 14:57:59.524300 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.524158    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7" event={"ID":"7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e","Type":"ContainerDied","Data":"f8ed3556fa58a0bbb7123fca04b4dd6623ea6fab7c97c2a1ec15f736c33697b8"}
Jul 21 14:57:59.524300 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.524188    2569 scope.go:117] "RemoveContainer" containerID="46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70"
Jul 21 14:57:59.533187 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.533167    2569 scope.go:117] "RemoveContainer" containerID="46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70"
Jul 21 14:57:59.533522 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:57:59.533502    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70\": container with ID starting with 46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70 not found: ID does not exist" containerID="46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70"
Jul 21 14:57:59.533605 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.533530    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70"} err="failed to get container status \"46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70\": rpc error: code = NotFound desc = could not find container \"46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70\": container with ID starting with 46ba75f69609531a9d4842d2f4a088ed88a21f9e6276c40de47d6b1c6c7d2e70 not found: ID does not exist"
Jul 21 14:57:59.546575 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.546553    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"]
Jul 21 14:57:59.551164 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:57:59.551136    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6ecdf5-576ff5ff6c-pthv7"]
Jul 21 14:58:00.467932 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:00.467898    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" path="/var/lib/kubelet/pods/7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e/volumes"
Jul 21 14:58:04.517036 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:04.517000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:58:26.839040 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:26.838943    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"]
Jul 21 14:58:26.839549 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:26.839215    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g" podUID="19ed5989-d23a-4bc7-b42d-b5dd08e93840" containerName="maas-api" containerID="cri-o://da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199" gracePeriod=30
Jul 21 14:58:27.087605 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.087579    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:58:27.155324 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.155248    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfqtp\" (UniqueName: \"kubernetes.io/projected/19ed5989-d23a-4bc7-b42d-b5dd08e93840-kube-api-access-pfqtp\") pod \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") "
Jul 21 14:58:27.155324 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.155317    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ed5989-d23a-4bc7-b42d-b5dd08e93840-openshift-service-ca\") pod \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") "
Jul 21 14:58:27.155538 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.155372    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19ed5989-d23a-4bc7-b42d-b5dd08e93840-maas-api-tls\") pod \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\" (UID: \"19ed5989-d23a-4bc7-b42d-b5dd08e93840\") "
Jul 21 14:58:27.155675 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.155654    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19ed5989-d23a-4bc7-b42d-b5dd08e93840-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19ed5989-d23a-4bc7-b42d-b5dd08e93840" (UID: "19ed5989-d23a-4bc7-b42d-b5dd08e93840"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:58:27.157605 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.157565    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19ed5989-d23a-4bc7-b42d-b5dd08e93840-kube-api-access-pfqtp" (OuterVolumeSpecName: "kube-api-access-pfqtp") pod "19ed5989-d23a-4bc7-b42d-b5dd08e93840" (UID: "19ed5989-d23a-4bc7-b42d-b5dd08e93840"). InnerVolumeSpecName "kube-api-access-pfqtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:58:27.157605 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.157585    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19ed5989-d23a-4bc7-b42d-b5dd08e93840-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19ed5989-d23a-4bc7-b42d-b5dd08e93840" (UID: "19ed5989-d23a-4bc7-b42d-b5dd08e93840"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:58:27.256463 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.256367    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfqtp\" (UniqueName: \"kubernetes.io/projected/19ed5989-d23a-4bc7-b42d-b5dd08e93840-kube-api-access-pfqtp\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:58:27.256463 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.256411    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ed5989-d23a-4bc7-b42d-b5dd08e93840-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:58:27.256463 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.256446    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19ed5989-d23a-4bc7-b42d-b5dd08e93840-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 14:58:27.645299 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.645265    2569 generic.go:356] "Generic (PLEG): container finished" podID="19ed5989-d23a-4bc7-b42d-b5dd08e93840" containerID="da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199" exitCode=0
Jul 21 14:58:27.645506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.645319    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"
Jul 21 14:58:27.645506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.645319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g" event={"ID":"19ed5989-d23a-4bc7-b42d-b5dd08e93840","Type":"ContainerDied","Data":"da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199"}
Jul 21 14:58:27.645506 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.645489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g" event={"ID":"19ed5989-d23a-4bc7-b42d-b5dd08e93840","Type":"ContainerDied","Data":"d787815ee314586061763870687d75a4c7cb3c63f0e83bbf2773e745b788944a"}
Jul 21 14:58:27.645692 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.645526    2569 scope.go:117] "RemoveContainer" containerID="da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199"
Jul 21 14:58:27.654690 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.654668    2569 scope.go:117] "RemoveContainer" containerID="da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199"
Jul 21 14:58:27.654942 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:58:27.654924    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199\": container with ID starting with da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199 not found: ID does not exist" containerID="da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199"
Jul 21 14:58:27.654988 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.654951    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199"} err="failed to get container status \"da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199\": rpc error: code = NotFound desc = could not find container \"da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199\": container with ID starting with da64811b6b704fd3d6673d4e8fbde8c9f1d36984267ed29d47455c8e21bfd199 not found: ID does not exist"
Jul 21 14:58:27.668526 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.668496    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"]
Jul 21 14:58:27.673464 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:27.673440    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-63bc89-858768bf88-m768g"]
Jul 21 14:58:28.468953 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:28.468915    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19ed5989-d23a-4bc7-b42d-b5dd08e93840" path="/var/lib/kubelet/pods/19ed5989-d23a-4bc7-b42d-b5dd08e93840/volumes"
Jul 21 14:58:31.225527 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:31.225488    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:58:43.033017 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.032978    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"]
Jul 21 14:58:43.033487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.033458    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" containerName="maas-api"
Jul 21 14:58:43.033487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.033474    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" containerName="maas-api"
Jul 21 14:58:43.033487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.033486    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ed5989-d23a-4bc7-b42d-b5dd08e93840" containerName="maas-api"
Jul 21 14:58:43.033595 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.033496    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ed5989-d23a-4bc7-b42d-b5dd08e93840" containerName="maas-api"
Jul 21 14:58:43.033595 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.033568    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ed5989-d23a-4bc7-b42d-b5dd08e93840" containerName="maas-api"
Jul 21 14:58:43.033595 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.033584    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7af9e7fd-0cd6-4285-b87c-bcec8fc2ff2e" containerName="maas-api"
Jul 21 14:58:43.036968 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.036948    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.039500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.039483    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:58:43.040027 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.040001    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bm4cc\""
Jul 21 14:58:43.040027 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.040016    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:58:43.040179 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.040010    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d89949-serving-cert\""
Jul 21 14:58:43.054982 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.054959    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"]
Jul 21 14:58:43.089173 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.089136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.089173 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.089174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.089385 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.089204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4684\" (UniqueName: \"kubernetes.io/projected/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-kube-api-access-g4684\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.190158 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.190124    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.190158 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.190157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.190448 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.190178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4684\" (UniqueName: \"kubernetes.io/projected/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-kube-api-access-g4684\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.190448 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:58:43.190267    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-serving-cert: secret "maas-api-e2e-shared-a-d89949-serving-cert" not found
Jul 21 14:58:43.190448 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:58:43.190339    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls podName:0b460fef-4ea0-4d9c-b1e7-c4de725d1e07 nodeName:}" failed. No retries permitted until 2026-07-21 14:58:43.690321911 +0000 UTC m=+2449.824717266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls") pod "maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" (UID: "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07") : secret "maas-api-e2e-shared-a-d89949-serving-cert" not found
Jul 21 14:58:43.190903 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.190880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.199806 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.199787    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4684\" (UniqueName: \"kubernetes.io/projected/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-kube-api-access-g4684\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.695117 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.695080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.697642 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.697621    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls\") pod \"maas-api-e2e-shared-a-d89949-695b4956dd-tbm95\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:43.947390 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:43.947307    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:44.088438 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:44.086261    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"]
Jul 21 14:58:44.091547 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:58:44.091515    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b460fef_4ea0_4d9c_b1e7_c4de725d1e07.slice/crio-f5fdef5f56fc8e44ac27df29341108fab00396a5837db8d752669673455534e6 WatchSource:0}: Error finding container f5fdef5f56fc8e44ac27df29341108fab00396a5837db8d752669673455534e6: Status 404 returned error can't find the container with id f5fdef5f56fc8e44ac27df29341108fab00396a5837db8d752669673455534e6
Jul 21 14:58:44.721659 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:44.721620    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" event={"ID":"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07","Type":"ContainerStarted","Data":"458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0"}
Jul 21 14:58:44.721846 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:44.721670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" event={"ID":"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07","Type":"ContainerStarted","Data":"f5fdef5f56fc8e44ac27df29341108fab00396a5837db8d752669673455534e6"}
Jul 21 14:58:44.721846 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:44.721719    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:44.747581 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:44.747529    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" podStartSLOduration=1.243631507 podStartE2EDuration="1.747512621s" totalImagesPullingTime="503.881114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:58:44.745705837 +0000 UTC m=+2450.880101211" watchObservedRunningTime="2026-07-21 14:58:44.747512621 +0000 UTC m=+2450.881907995"
Jul 21 14:58:48.544729 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:48.544691    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:58:50.732149 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:50.732119    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 14:58:50.975938 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:50.975909    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"]
Jul 21 14:58:50.985171 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:50.985094    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:50.988449 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:50.988413    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-449411-serving-cert\""
Jul 21 14:58:50.994226 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:50.994201    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"]
Jul 21 14:58:51.061668 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.061630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.061668 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.061669    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b101d-4472-4c35-9d83-240e34cba734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.061897 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.061734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f9h7\" (UniqueName: \"kubernetes.io/projected/2c0b101d-4472-4c35-9d83-240e34cba734-kube-api-access-2f9h7\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.162929 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.162888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.162929 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.162931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b101d-4472-4c35-9d83-240e34cba734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.163184 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.162961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f9h7\" (UniqueName: \"kubernetes.io/projected/2c0b101d-4472-4c35-9d83-240e34cba734-kube-api-access-2f9h7\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.163184 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:58:51.163064    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-serving-cert: secret "maas-api-e2e-shared-b-449411-serving-cert" not found
Jul 21 14:58:51.163184 ip-10-0-129-87 kubenswrapper[2569]: E0721 14:58:51.163137    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls podName:2c0b101d-4472-4c35-9d83-240e34cba734 nodeName:}" failed. No retries permitted until 2026-07-21 14:58:51.663120216 +0000 UTC m=+2457.797515568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls") pod "maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" (UID: "2c0b101d-4472-4c35-9d83-240e34cba734") : secret "maas-api-e2e-shared-b-449411-serving-cert" not found
Jul 21 14:58:51.163680 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.163656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b101d-4472-4c35-9d83-240e34cba734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.172098 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.172078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f9h7\" (UniqueName: \"kubernetes.io/projected/2c0b101d-4472-4c35-9d83-240e34cba734-kube-api-access-2f9h7\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.667500 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.667462    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.670037 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.670013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls\") pod \"maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:51.895640 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:51.895592    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:52.050134 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:52.050101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"]
Jul 21 14:58:52.053353 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:58:52.053328    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0b101d_4472_4c35_9d83_240e34cba734.slice/crio-75c279e8e7b0adedcd1bb9cc5895662c22a95a5fb9d5a666ef53b0651effa986 WatchSource:0}: Error finding container 75c279e8e7b0adedcd1bb9cc5895662c22a95a5fb9d5a666ef53b0651effa986: Status 404 returned error can't find the container with id 75c279e8e7b0adedcd1bb9cc5895662c22a95a5fb9d5a666ef53b0651effa986
Jul 21 14:58:52.757130 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:52.757091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" event={"ID":"2c0b101d-4472-4c35-9d83-240e34cba734","Type":"ContainerStarted","Data":"37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2"}
Jul 21 14:58:52.757296 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:52.757154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" event={"ID":"2c0b101d-4472-4c35-9d83-240e34cba734","Type":"ContainerStarted","Data":"75c279e8e7b0adedcd1bb9cc5895662c22a95a5fb9d5a666ef53b0651effa986"}
Jul 21 14:58:52.757296 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:52.757206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:58.767560 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:58.767529    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 14:58:58.804155 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:58.804106    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" podStartSLOduration=8.431281593 podStartE2EDuration="8.804089082s" totalImagesPullingTime="372.807489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:58:52.788300266 +0000 UTC m=+2458.922695640" watchObservedRunningTime="2026-07-21 14:58:58.804089082 +0000 UTC m=+2464.938484456"
Jul 21 14:58:59.872213 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:58:59.872179    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:59:22.331755 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:22.331719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:59:35.653201 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.653170    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"]
Jul 21 14:59:35.656557 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.656536    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.659202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.659178    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449411\"/\"auth-test-model-449411-kserve-self-signed-certs\""
Jul 21 14:59:35.659202 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.659178    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449411\"/\"kube-root-ca.crt\""
Jul 21 14:59:35.659378 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.659206    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449411\"/\"openshift-service-ca.crt\""
Jul 21 14:59:35.659378 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.659187    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449411\"/\"default-dockercfg-jw7fr\""
Jul 21 14:59:35.670288 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.670263    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"]
Jul 21 14:59:35.752022 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.751989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-home\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.752022 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.752022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-dshm\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.752237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.752060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tls-certs\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.752237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.752127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27bhs\" (UniqueName: \"kubernetes.io/projected/433643a8-3dd3-4fbd-b66b-a7fec9d50032-kube-api-access-27bhs\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.752237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.752197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-model-cache\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.752237 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.752235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tmp-dir\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.852723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.852683    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-home\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.852723 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.852720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-dshm\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.852967 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.852753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tls-certs\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.852967 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.852872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27bhs\" (UniqueName: \"kubernetes.io/projected/433643a8-3dd3-4fbd-b66b-a7fec9d50032-kube-api-access-27bhs\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.852967 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.852906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-model-cache\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.852967 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.852950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tmp-dir\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.853253 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.853141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-home\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.853320 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.853301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-model-cache\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.853365 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.853332    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tmp-dir\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.855234 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.855204    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-dshm\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.855487 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.855471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tls-certs\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.864818 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.864799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27bhs\" (UniqueName: \"kubernetes.io/projected/433643a8-3dd3-4fbd-b66b-a7fec9d50032-kube-api-access-27bhs\") pod \"auth-test-model-449411-kserve-78f998767f-62sz8\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") " pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:35.965835 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:35.965802    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:36.107377 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:36.107348    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"]
Jul 21 14:59:36.110431 ip-10-0-129-87 kubenswrapper[2569]: W0721 14:59:36.110387    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod433643a8_3dd3_4fbd_b66b_a7fec9d50032.slice/crio-936c6e5ef0ace3712f9a215b0a243c090f8a3b721431a01fd99a773fc1212c9f WatchSource:0}: Error finding container 936c6e5ef0ace3712f9a215b0a243c090f8a3b721431a01fd99a773fc1212c9f: Status 404 returned error can't find the container with id 936c6e5ef0ace3712f9a215b0a243c090f8a3b721431a01fd99a773fc1212c9f
Jul 21 14:59:36.949308 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:36.949269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8" event={"ID":"433643a8-3dd3-4fbd-b66b-a7fec9d50032","Type":"ContainerStarted","Data":"e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24"}
Jul 21 14:59:36.949715 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:36.949316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8" event={"ID":"433643a8-3dd3-4fbd-b66b-a7fec9d50032","Type":"ContainerStarted","Data":"936c6e5ef0ace3712f9a215b0a243c090f8a3b721431a01fd99a773fc1212c9f"}
Jul 21 14:59:36.970368 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:36.970322    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8" podStartSLOduration=1.970309341 podStartE2EDuration="1.970309341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:59:36.968976776 +0000 UTC m=+2503.103372150" watchObservedRunningTime="2026-07-21 14:59:36.970309341 +0000 UTC m=+2503.104704715"
Jul 21 14:59:39.327634 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:39.327599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 14:59:45.966559 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:45.966466    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:45.966559 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:45.966518    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:45.979143 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:45.979117    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:46.002967 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:46.002942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 14:59:53.442389 ip-10-0-129-87 kubenswrapper[2569]: I0721 14:59:53.442352    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:00:09.383377 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:00:09.383340    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:00:25.736340 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:00:25.736305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:00:43.618816 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:00:43.618782    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:01:00.343527 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:00.343494    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:01:17.247681 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:17.247600    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:01:32.751553 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:32.751523    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"]
Jul 21 15:01:34.428586 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:34.428541    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:01:47.771407 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:47.771346    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8" podUID="433643a8-3dd3-4fbd-b66b-a7fec9d50032" containerName="main" containerID="cri-o://e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24" gracePeriod=45
Jul 21 15:01:48.022540 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.022513    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 15:01:48.057704 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057679    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tmp-dir\") pod \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") "
Jul 21 15:01:48.057858 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057721    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-home\") pod \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") "
Jul 21 15:01:48.057858 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057744    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-dshm\") pod \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") "
Jul 21 15:01:48.057858 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057768    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tls-certs\") pod \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") "
Jul 21 15:01:48.057858 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057791    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-model-cache\") pod \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") "
Jul 21 15:01:48.057858 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27bhs\" (UniqueName: \"kubernetes.io/projected/433643a8-3dd3-4fbd-b66b-a7fec9d50032-kube-api-access-27bhs\") pod \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\" (UID: \"433643a8-3dd3-4fbd-b66b-a7fec9d50032\") "
Jul 21 15:01:48.058120 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.058008    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-home" (OuterVolumeSpecName: "home") pod "433643a8-3dd3-4fbd-b66b-a7fec9d50032" (UID: "433643a8-3dd3-4fbd-b66b-a7fec9d50032"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:01:48.058120 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.057997    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "433643a8-3dd3-4fbd-b66b-a7fec9d50032" (UID: "433643a8-3dd3-4fbd-b66b-a7fec9d50032"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:01:48.058120 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.058035    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-model-cache" (OuterVolumeSpecName: "model-cache") pod "433643a8-3dd3-4fbd-b66b-a7fec9d50032" (UID: "433643a8-3dd3-4fbd-b66b-a7fec9d50032"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:01:48.058120 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.058094    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-home\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:01:48.060233 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.060203    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "433643a8-3dd3-4fbd-b66b-a7fec9d50032" (UID: "433643a8-3dd3-4fbd-b66b-a7fec9d50032"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:01:48.060233 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.060221    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/433643a8-3dd3-4fbd-b66b-a7fec9d50032-kube-api-access-27bhs" (OuterVolumeSpecName: "kube-api-access-27bhs") pod "433643a8-3dd3-4fbd-b66b-a7fec9d50032" (UID: "433643a8-3dd3-4fbd-b66b-a7fec9d50032"). InnerVolumeSpecName "kube-api-access-27bhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:01:48.060233 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.060228    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-dshm" (OuterVolumeSpecName: "dshm") pod "433643a8-3dd3-4fbd-b66b-a7fec9d50032" (UID: "433643a8-3dd3-4fbd-b66b-a7fec9d50032"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:01:48.158805 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.158771    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-dshm\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:01:48.158805 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.158797    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tls-certs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:01:48.158805 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.158808    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-model-cache\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:01:48.159045 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.158815    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27bhs\" (UniqueName: \"kubernetes.io/projected/433643a8-3dd3-4fbd-b66b-a7fec9d50032-kube-api-access-27bhs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:01:48.159045 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.158824    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433643a8-3dd3-4fbd-b66b-a7fec9d50032-tmp-dir\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:01:48.529476 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.529436    2569 generic.go:356] "Generic (PLEG): container finished" podID="433643a8-3dd3-4fbd-b66b-a7fec9d50032" containerID="e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24" exitCode=0
Jul 21 15:01:48.529644 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.529500    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"
Jul 21 15:01:48.529644 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.529514    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8" event={"ID":"433643a8-3dd3-4fbd-b66b-a7fec9d50032","Type":"ContainerDied","Data":"e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24"}
Jul 21 15:01:48.529644 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.529559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8" event={"ID":"433643a8-3dd3-4fbd-b66b-a7fec9d50032","Type":"ContainerDied","Data":"936c6e5ef0ace3712f9a215b0a243c090f8a3b721431a01fd99a773fc1212c9f"}
Jul 21 15:01:48.529644 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.529576    2569 scope.go:117] "RemoveContainer" containerID="e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24"
Jul 21 15:01:48.538202 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.538185    2569 scope.go:117] "RemoveContainer" containerID="e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24"
Jul 21 15:01:48.538501 ip-10-0-129-87 kubenswrapper[2569]: E0721 15:01:48.538481    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24\": container with ID starting with e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24 not found: ID does not exist" containerID="e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24"
Jul 21 15:01:48.538554 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.538508    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24"} err="failed to get container status \"e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24\": rpc error: code = NotFound desc = could not find container \"e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24\": container with ID starting with e91df4b6ecf8bd09169fffe1e1e62d8e82f0709390186afbbe85bfb3b8bcde24 not found: ID does not exist"
Jul 21 15:01:48.548554 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.548527    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"]
Jul 21 15:01:48.553760 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:48.553733    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-449411/auth-test-model-449411-kserve-78f998767f-62sz8"]
Jul 21 15:01:50.467875 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:50.467845    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="433643a8-3dd3-4fbd-b66b-a7fec9d50032" path="/var/lib/kubelet/pods/433643a8-3dd3-4fbd-b66b-a7fec9d50032/volumes"
Jul 21 15:01:51.727010 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:01:51.726976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:02:23.425236 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:02:23.425198    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:02:45.430815 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:02:45.430729    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:03:07.922014 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:07.921976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:03:15.059527 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.059499    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"]
Jul 21 15:03:15.059891 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.059822    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="433643a8-3dd3-4fbd-b66b-a7fec9d50032" containerName="main"
Jul 21 15:03:15.059891 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.059832    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="433643a8-3dd3-4fbd-b66b-a7fec9d50032" containerName="main"
Jul 21 15:03:15.059967 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.059899    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="433643a8-3dd3-4fbd-b66b-a7fec9d50032" containerName="main"
Jul 21 15:03:15.062910 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.062893    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.065921 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.065886    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d89949\"/\"rate-test-model-d89949-kserve-self-signed-certs\""
Jul 21 15:03:15.066061 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.065932    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d89949\"/\"kube-root-ca.crt\""
Jul 21 15:03:15.066061 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.065959    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d89949\"/\"openshift-service-ca.crt\""
Jul 21 15:03:15.066061 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.066038    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d89949\"/\"default-dockercfg-9cmfc\""
Jul 21 15:03:15.073198 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.073167    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"]
Jul 21 15:03:15.130322 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.130283    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-dshm\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.130540 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.130358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-home\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.130540 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.130395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-model-cache\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.130540 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.130450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591e299-fc21-419f-87e4-c2854600c8d5-tls-certs\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.130540 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.130473    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6jbt\" (UniqueName: \"kubernetes.io/projected/6591e299-fc21-419f-87e4-c2854600c8d5-kube-api-access-r6jbt\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.130717 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.130556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-tmp-dir\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231231 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-home\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231231 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-model-cache\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231511 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591e299-fc21-419f-87e4-c2854600c8d5-tls-certs\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231511 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231276    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6jbt\" (UniqueName: \"kubernetes.io/projected/6591e299-fc21-419f-87e4-c2854600c8d5-kube-api-access-r6jbt\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231511 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-tmp-dir\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231511 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-dshm\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231722 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231674    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-home\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231722 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231689    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-tmp-dir\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.231818 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.231736    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-model-cache\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.233748 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.233726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-dshm\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.234053 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.234037    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591e299-fc21-419f-87e4-c2854600c8d5-tls-certs\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.242474 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.242449    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6jbt\" (UniqueName: \"kubernetes.io/projected/6591e299-fc21-419f-87e4-c2854600c8d5-kube-api-access-r6jbt\") pod \"rate-test-model-d89949-kserve-5c77dd697f-cxnt8\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") " pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.372497 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.372402    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:15.514328 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.514296    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"]
Jul 21 15:03:15.517387 ip-10-0-129-87 kubenswrapper[2569]: W0721 15:03:15.517359    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6591e299_fc21_419f_87e4_c2854600c8d5.slice/crio-5f00e7e5e8e8af0683033b2efda9cf1ed9f4a5cc485f08fc4d3389e039d98f40 WatchSource:0}: Error finding container 5f00e7e5e8e8af0683033b2efda9cf1ed9f4a5cc485f08fc4d3389e039d98f40: Status 404 returned error can't find the container with id 5f00e7e5e8e8af0683033b2efda9cf1ed9f4a5cc485f08fc4d3389e039d98f40
Jul 21 15:03:15.916337 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.916301    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8" event={"ID":"6591e299-fc21-419f-87e4-c2854600c8d5","Type":"ContainerStarted","Data":"12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2"}
Jul 21 15:03:15.916337 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.916342    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8" event={"ID":"6591e299-fc21-419f-87e4-c2854600c8d5","Type":"ContainerStarted","Data":"5f00e7e5e8e8af0683033b2efda9cf1ed9f4a5cc485f08fc4d3389e039d98f40"}
Jul 21 15:03:15.937544 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:15.937488    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8" podStartSLOduration=0.937470694 podStartE2EDuration="937.470694ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:03:15.936349449 +0000 UTC m=+2722.070744823" watchObservedRunningTime="2026-07-21 15:03:15.937470694 +0000 UTC m=+2722.071866107"
Jul 21 15:03:20.453730 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:20.453695    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:03:25.373111 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:25.373081    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:25.373556 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:25.373125    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:25.385613 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:25.385589    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:25.969759 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:25.969730    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:03:48.018630 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:03:48.018528    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:04:06.124224 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:06.124186    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:04:20.824849 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:20.824768    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:04:27.851776 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:27.851738    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"]
Jul 21 15:04:40.935601 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:40.935563    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:04:42.872228 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:42.872184    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8" podUID="6591e299-fc21-419f-87e4-c2854600c8d5" containerName="main" containerID="cri-o://12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2" gracePeriod=45
Jul 21 15:04:43.120690 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.120666    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:04:43.209139 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209109    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6jbt\" (UniqueName: \"kubernetes.io/projected/6591e299-fc21-419f-87e4-c2854600c8d5-kube-api-access-r6jbt\") pod \"6591e299-fc21-419f-87e4-c2854600c8d5\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") "
Jul 21 15:04:43.209330 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-dshm\") pod \"6591e299-fc21-419f-87e4-c2854600c8d5\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") "
Jul 21 15:04:43.209330 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209208    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-home\") pod \"6591e299-fc21-419f-87e4-c2854600c8d5\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") "
Jul 21 15:04:43.209330 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209301    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-model-cache\") pod \"6591e299-fc21-419f-87e4-c2854600c8d5\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") "
Jul 21 15:04:43.209538 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209334    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591e299-fc21-419f-87e4-c2854600c8d5-tls-certs\") pod \"6591e299-fc21-419f-87e4-c2854600c8d5\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") "
Jul 21 15:04:43.209538 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209370    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-tmp-dir\") pod \"6591e299-fc21-419f-87e4-c2854600c8d5\" (UID: \"6591e299-fc21-419f-87e4-c2854600c8d5\") "
Jul 21 15:04:43.209538 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209471    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-home" (OuterVolumeSpecName: "home") pod "6591e299-fc21-419f-87e4-c2854600c8d5" (UID: "6591e299-fc21-419f-87e4-c2854600c8d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:04:43.209669 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209580    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "6591e299-fc21-419f-87e4-c2854600c8d5" (UID: "6591e299-fc21-419f-87e4-c2854600c8d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:04:43.209710 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209698    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-home\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:43.209754 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209712    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-model-cache\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:43.209816 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.209789    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6591e299-fc21-419f-87e4-c2854600c8d5" (UID: "6591e299-fc21-419f-87e4-c2854600c8d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:04:43.211683 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.211650    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6591e299-fc21-419f-87e4-c2854600c8d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6591e299-fc21-419f-87e4-c2854600c8d5" (UID: "6591e299-fc21-419f-87e4-c2854600c8d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:04:43.211786 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.211708    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6591e299-fc21-419f-87e4-c2854600c8d5-kube-api-access-r6jbt" (OuterVolumeSpecName: "kube-api-access-r6jbt") pod "6591e299-fc21-419f-87e4-c2854600c8d5" (UID: "6591e299-fc21-419f-87e4-c2854600c8d5"). InnerVolumeSpecName "kube-api-access-r6jbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:04:43.211786 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.211767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-dshm" (OuterVolumeSpecName: "dshm") pod "6591e299-fc21-419f-87e4-c2854600c8d5" (UID: "6591e299-fc21-419f-87e4-c2854600c8d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:04:43.304941 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.304901    2569 generic.go:356] "Generic (PLEG): container finished" podID="6591e299-fc21-419f-87e4-c2854600c8d5" containerID="12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2" exitCode=0
Jul 21 15:04:43.304941 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.304944    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"
Jul 21 15:04:43.305197 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.304951    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8" event={"ID":"6591e299-fc21-419f-87e4-c2854600c8d5","Type":"ContainerDied","Data":"12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2"}
Jul 21 15:04:43.305197 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.304981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8" event={"ID":"6591e299-fc21-419f-87e4-c2854600c8d5","Type":"ContainerDied","Data":"5f00e7e5e8e8af0683033b2efda9cf1ed9f4a5cc485f08fc4d3389e039d98f40"}
Jul 21 15:04:43.305197 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.304999    2569 scope.go:117] "RemoveContainer" containerID="12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2"
Jul 21 15:04:43.310728 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.310708    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6jbt\" (UniqueName: \"kubernetes.io/projected/6591e299-fc21-419f-87e4-c2854600c8d5-kube-api-access-r6jbt\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:43.310827 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.310733    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-dshm\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:43.310827 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.310745    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591e299-fc21-419f-87e4-c2854600c8d5-tls-certs\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:43.310827 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.310773    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6591e299-fc21-419f-87e4-c2854600c8d5-tmp-dir\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:43.314040 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.314020    2569 scope.go:117] "RemoveContainer" containerID="12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2"
Jul 21 15:04:43.314291 ip-10-0-129-87 kubenswrapper[2569]: E0721 15:04:43.314273    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2\": container with ID starting with 12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2 not found: ID does not exist" containerID="12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2"
Jul 21 15:04:43.314352 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.314303    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2"} err="failed to get container status \"12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2\": rpc error: code = NotFound desc = could not find container \"12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2\": container with ID starting with 12a5cbe4300394f8f438cba9b21f132de6fb4b08d9c77a070680ea690ec7cad2 not found: ID does not exist"
Jul 21 15:04:43.328118 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.328097    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"]
Jul 21 15:04:43.339206 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:43.339181    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d89949/rate-test-model-d89949-kserve-5c77dd697f-cxnt8"]
Jul 21 15:04:44.468609 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:44.468566    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6591e299-fc21-419f-87e4-c2854600c8d5" path="/var/lib/kubelet/pods/6591e299-fc21-419f-87e4-c2854600c8d5/volumes"
Jul 21 15:04:58.640567 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.640534    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"]
Jul 21 15:04:58.640994 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.640779    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" podUID="0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" containerName="maas-api" containerID="cri-o://458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0" gracePeriod=30
Jul 21 15:04:58.882864 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.882840    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 15:04:58.943717 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.943684    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls\") pod \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") "
Jul 21 15:04:58.943885 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.943721    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4684\" (UniqueName: \"kubernetes.io/projected/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-kube-api-access-g4684\") pod \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") "
Jul 21 15:04:58.943885 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.943859    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-openshift-service-ca\") pod \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\" (UID: \"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07\") "
Jul 21 15:04:58.944171 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.944152    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" (UID: "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:04:58.945880 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.945864    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" (UID: "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:04:58.945970 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:58.945954    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-kube-api-access-g4684" (OuterVolumeSpecName: "kube-api-access-g4684") pod "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" (UID: "0b460fef-4ea0-4d9c-b1e7-c4de725d1e07"). InnerVolumeSpecName "kube-api-access-g4684". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:04:59.045388 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.045352    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:59.045388 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.045380    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:59.045388 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.045391    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4684\" (UniqueName: \"kubernetes.io/projected/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07-kube-api-access-g4684\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:04:59.375638 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.375550    2569 generic.go:356] "Generic (PLEG): container finished" podID="0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" containerID="458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0" exitCode=0
Jul 21 15:04:59.375638 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.375595    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"
Jul 21 15:04:59.375638 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.375624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" event={"ID":"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07","Type":"ContainerDied","Data":"458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0"}
Jul 21 15:04:59.375883 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.375690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95" event={"ID":"0b460fef-4ea0-4d9c-b1e7-c4de725d1e07","Type":"ContainerDied","Data":"f5fdef5f56fc8e44ac27df29341108fab00396a5837db8d752669673455534e6"}
Jul 21 15:04:59.375883 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.375714    2569 scope.go:117] "RemoveContainer" containerID="458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0"
Jul 21 15:04:59.385127 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.385111    2569 scope.go:117] "RemoveContainer" containerID="458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0"
Jul 21 15:04:59.385380 ip-10-0-129-87 kubenswrapper[2569]: E0721 15:04:59.385363    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0\": container with ID starting with 458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0 not found: ID does not exist" containerID="458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0"
Jul 21 15:04:59.385438 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.385389    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0"} err="failed to get container status \"458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0\": rpc error: code = NotFound desc = could not find container \"458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0\": container with ID starting with 458f013a1a3d3e9ab5fbcf42ea056cb463c581a360f0835d3bd527a9642924b0 not found: ID does not exist"
Jul 21 15:04:59.399862 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.399833    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"]
Jul 21 15:04:59.404834 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:04:59.404812    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d89949-695b4956dd-tbm95"]
Jul 21 15:05:00.467945 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:00.467914    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" path="/var/lib/kubelet/pods/0b460fef-4ea0-4d9c-b1e7-c4de725d1e07/volumes"
Jul 21 15:05:01.539239 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:01.539205    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:05:21.381517 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.381481    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"]
Jul 21 15:05:21.468606 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.468537    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" podUID="2c0b101d-4472-4c35-9d83-240e34cba734" containerName="maas-api" containerID="cri-o://37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2" gracePeriod=30
Jul 21 15:05:21.707816 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.707794    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 15:05:21.843617 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.843586    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls\") pod \"2c0b101d-4472-4c35-9d83-240e34cba734\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") "
Jul 21 15:05:21.843794 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.843677    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f9h7\" (UniqueName: \"kubernetes.io/projected/2c0b101d-4472-4c35-9d83-240e34cba734-kube-api-access-2f9h7\") pod \"2c0b101d-4472-4c35-9d83-240e34cba734\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") "
Jul 21 15:05:21.843794 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.843743    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b101d-4472-4c35-9d83-240e34cba734-openshift-service-ca\") pod \"2c0b101d-4472-4c35-9d83-240e34cba734\" (UID: \"2c0b101d-4472-4c35-9d83-240e34cba734\") "
Jul 21 15:05:21.844060 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.844032    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c0b101d-4472-4c35-9d83-240e34cba734-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c0b101d-4472-4c35-9d83-240e34cba734" (UID: "2c0b101d-4472-4c35-9d83-240e34cba734"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:05:21.845908 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.845887    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c0b101d-4472-4c35-9d83-240e34cba734" (UID: "2c0b101d-4472-4c35-9d83-240e34cba734"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:05:21.845908 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.845889    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0b101d-4472-4c35-9d83-240e34cba734-kube-api-access-2f9h7" (OuterVolumeSpecName: "kube-api-access-2f9h7") pod "2c0b101d-4472-4c35-9d83-240e34cba734" (UID: "2c0b101d-4472-4c35-9d83-240e34cba734"). InnerVolumeSpecName "kube-api-access-2f9h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:05:21.944922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.944896    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f9h7\" (UniqueName: \"kubernetes.io/projected/2c0b101d-4472-4c35-9d83-240e34cba734-kube-api-access-2f9h7\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:05:21.944922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.944924    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b101d-4472-4c35-9d83-240e34cba734-openshift-service-ca\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:05:21.945112 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:21.944935    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0b101d-4472-4c35-9d83-240e34cba734-maas-api-tls\") on node \"ip-10-0-129-87.ec2.internal\" DevicePath \"\""
Jul 21 15:05:22.474658 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.474622    2569 generic.go:356] "Generic (PLEG): container finished" podID="2c0b101d-4472-4c35-9d83-240e34cba734" containerID="37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2" exitCode=0
Jul 21 15:05:22.475087 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.474664    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"
Jul 21 15:05:22.475087 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.474707    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" event={"ID":"2c0b101d-4472-4c35-9d83-240e34cba734","Type":"ContainerDied","Data":"37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2"}
Jul 21 15:05:22.475087 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.474755    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8" event={"ID":"2c0b101d-4472-4c35-9d83-240e34cba734","Type":"ContainerDied","Data":"75c279e8e7b0adedcd1bb9cc5895662c22a95a5fb9d5a666ef53b0651effa986"}
Jul 21 15:05:22.475087 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.474777    2569 scope.go:117] "RemoveContainer" containerID="37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2"
Jul 21 15:05:22.484134 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.484116    2569 scope.go:117] "RemoveContainer" containerID="37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2"
Jul 21 15:05:22.484512 ip-10-0-129-87 kubenswrapper[2569]: E0721 15:05:22.484402    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2\": container with ID starting with 37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2 not found: ID does not exist" containerID="37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2"
Jul 21 15:05:22.484512 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.484446    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2"} err="failed to get container status \"37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2\": rpc error: code = NotFound desc = could not find container \"37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2\": container with ID starting with 37b65b7ed85d4de7c77246787ba2ed4f628255e2655f3f05825948de747cd9c2 not found: ID does not exist"
Jul 21 15:05:22.496873 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.496849    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"]
Jul 21 15:05:22.501770 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:22.501746    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-449411-55fd9fb49c-xwkm8"]
Jul 21 15:05:24.346000 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:24.345965    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4sj9s"]
Jul 21 15:05:24.472667 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:24.472636    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c0b101d-4472-4c35-9d83-240e34cba734" path="/var/lib/kubelet/pods/2c0b101d-4472-4c35-9d83-240e34cba734/volumes"
Jul 21 15:05:34.864479 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:34.864441    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b9b8b88b8-fkhkz_08e2c2d2-70f6-448f-bd8c-049d34de961c/authorino/0.log"
Jul 21 15:05:40.593039 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:40.593006    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wwjvz_40c27eae-2b98-4af8-bc7b-7ae6a83b4529/manager/0.log"
Jul 21 15:05:41.319336 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:41.319308    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bb54d755-wrxfj_8e662208-e323-4a0f-b7ad-8b70cc0c8818/copy-manifests/0.log"
Jul 21 15:05:41.327876 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:41.327848    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bb54d755-wrxfj_8e662208-e323-4a0f-b7ad-8b70cc0c8818/manager/0.log"
Jul 21 15:05:43.609363 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:43.609332    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b9b8b88b8-fkhkz_08e2c2d2-70f6-448f-bd8c-049d34de961c/authorino/0.log"
Jul 21 15:05:43.964665 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:43.964640    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-v4c47_4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7/kuadrant-console-plugin/0.log"
Jul 21 15:05:44.338970 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:44.338860    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4sj9s_8cbcb661-8caf-47d0-b274-d9fff98edaf8/limitador/0.log"
Jul 21 15:05:45.081626 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:45.081597    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd655967-thj96_dc629773-d072-453c-86ef-b5843a33fb3f/kube-auth-proxy/0.log"
Jul 21 15:05:46.669513 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:46.669458    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-228x4_75b67941-17a2-4c98-9849-8f162039d05b/main/0.log"
Jul 21 15:05:52.693091 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693056    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5jhqw/must-gather-mjbpf"]
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693349    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" containerName="maas-api"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693359    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" containerName="maas-api"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693376    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6591e299-fc21-419f-87e4-c2854600c8d5" containerName="main"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693381    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6591e299-fc21-419f-87e4-c2854600c8d5" containerName="main"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693388    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0b101d-4472-4c35-9d83-240e34cba734" containerName="maas-api"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693394    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0b101d-4472-4c35-9d83-240e34cba734" containerName="maas-api"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693476    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6591e299-fc21-419f-87e4-c2854600c8d5" containerName="main"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693486    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0b101d-4472-4c35-9d83-240e34cba734" containerName="maas-api"
Jul 21 15:05:52.693509 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.693497    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b460fef-4ea0-4d9c-b1e7-c4de725d1e07" containerName="maas-api"
Jul 21 15:05:52.696478 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.696456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:52.698892 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.698859    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5jhqw\"/\"openshift-service-ca.crt\""
Jul 21 15:05:52.699032 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.699008    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5jhqw\"/\"kube-root-ca.crt\""
Jul 21 15:05:52.699506 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.699491    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5jhqw\"/\"default-dockercfg-lz5ll\""
Jul 21 15:05:52.707951 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.707931    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5jhqw/must-gather-mjbpf"]
Jul 21 15:05:52.805912 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.805876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/baa440d1-4d21-401c-8aee-847e18be64e4-must-gather-output\") pod \"must-gather-mjbpf\" (UID: \"baa440d1-4d21-401c-8aee-847e18be64e4\") " pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:52.806099 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.805922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wvnn\" (UniqueName: \"kubernetes.io/projected/baa440d1-4d21-401c-8aee-847e18be64e4-kube-api-access-2wvnn\") pod \"must-gather-mjbpf\" (UID: \"baa440d1-4d21-401c-8aee-847e18be64e4\") " pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:52.907195 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.907158    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/baa440d1-4d21-401c-8aee-847e18be64e4-must-gather-output\") pod \"must-gather-mjbpf\" (UID: \"baa440d1-4d21-401c-8aee-847e18be64e4\") " pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:52.907195 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.907202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wvnn\" (UniqueName: \"kubernetes.io/projected/baa440d1-4d21-401c-8aee-847e18be64e4-kube-api-access-2wvnn\") pod \"must-gather-mjbpf\" (UID: \"baa440d1-4d21-401c-8aee-847e18be64e4\") " pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:52.907517 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.907487    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/baa440d1-4d21-401c-8aee-847e18be64e4-must-gather-output\") pod \"must-gather-mjbpf\" (UID: \"baa440d1-4d21-401c-8aee-847e18be64e4\") " pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:52.916395 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:52.916370    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wvnn\" (UniqueName: \"kubernetes.io/projected/baa440d1-4d21-401c-8aee-847e18be64e4-kube-api-access-2wvnn\") pod \"must-gather-mjbpf\" (UID: \"baa440d1-4d21-401c-8aee-847e18be64e4\") " pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:53.005657 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:53.005543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5jhqw/must-gather-mjbpf"
Jul 21 15:05:53.148938 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:53.148915    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5jhqw/must-gather-mjbpf"]
Jul 21 15:05:53.151414 ip-10-0-129-87 kubenswrapper[2569]: W0721 15:05:53.151387    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbaa440d1_4d21_401c_8aee_847e18be64e4.slice/crio-c2775195fe26ac78112c38a97098dd4cd6bb63de2e7ffa93c811854e1f5d051b WatchSource:0}: Error finding container c2775195fe26ac78112c38a97098dd4cd6bb63de2e7ffa93c811854e1f5d051b: Status 404 returned error can't find the container with id c2775195fe26ac78112c38a97098dd4cd6bb63de2e7ffa93c811854e1f5d051b
Jul 21 15:05:53.153181 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:53.153163    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 15:05:53.612649 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:53.612617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5jhqw/must-gather-mjbpf" event={"ID":"baa440d1-4d21-401c-8aee-847e18be64e4","Type":"ContainerStarted","Data":"c2775195fe26ac78112c38a97098dd4cd6bb63de2e7ffa93c811854e1f5d051b"}
Jul 21 15:05:55.622400 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:55.622369    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5jhqw/must-gather-mjbpf" event={"ID":"baa440d1-4d21-401c-8aee-847e18be64e4","Type":"ContainerStarted","Data":"9f64a1242c5ef64523cec0dc5749fd02e6d4a4d472a31f98dc471efdc6f91d1b"}
Jul 21 15:05:55.622841 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:55.622431    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5jhqw/must-gather-mjbpf" event={"ID":"baa440d1-4d21-401c-8aee-847e18be64e4","Type":"ContainerStarted","Data":"7daaf9aeaa1ba3556a3f82eaa1d22bd068333425815c568ef642599f0d61935e"}
Jul 21 15:05:55.647145 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:55.647044    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5jhqw/must-gather-mjbpf" podStartSLOduration=1.38665991 podStartE2EDuration="3.647012209s" totalImagesPullingTime="2.260352299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:05:55.645855756 +0000 UTC m=+2881.780251132" watchObservedRunningTime="2026-07-21 15:05:55.647012209 +0000 UTC m=+2881.781407583"
Jul 21 15:05:57.260182 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:57.260147    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-24x6m_55a3aa06-f532-4817-a3ad-dfbcf037ec5a/global-pull-secret-syncer/0.log"
Jul 21 15:05:57.518216 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:57.518140    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zfl49_ca341b8b-89f3-452c-ac8e-a2c40e16bf40/konnectivity-agent/0.log"
Jul 21 15:05:57.549583 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:05:57.549556    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-87.ec2.internal_775ad56364ad1161b542cba6cf3f9498/haproxy/0.log"
Jul 21 15:06:03.249399 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:03.248404    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b9b8b88b8-fkhkz_08e2c2d2-70f6-448f-bd8c-049d34de961c/authorino/0.log"
Jul 21 15:06:03.357958 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:03.357917    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-v4c47_4c19610a-f74d-40bd-bdd6-3f3f7f1f2ff7/kuadrant-console-plugin/0.log"
Jul 21 15:06:03.645686 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:03.645600    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4sj9s_8cbcb661-8caf-47d0-b274-d9fff98edaf8/limitador/0.log"
Jul 21 15:06:05.839880 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:05.839843    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-2zxbg_7c00c960-ce49-4140-95ac-fe0d22a01c97/kube-state-metrics/0.log"
Jul 21 15:06:05.864404 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:05.864377    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-2zxbg_7c00c960-ce49-4140-95ac-fe0d22a01c97/kube-rbac-proxy-main/0.log"
Jul 21 15:06:05.893195 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:05.893160    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-2zxbg_7c00c960-ce49-4140-95ac-fe0d22a01c97/kube-rbac-proxy-self/0.log"
Jul 21 15:06:05.931946 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:05.931921    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d86db7f6f-dwrc2_15449fe8-6595-4138-8a44-4e79336a42a9/metrics-server/0.log"
Jul 21 15:06:06.119260 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.119166    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qdd9p_8407a2ae-c501-42e8-ab18-68e02523212a/node-exporter/0.log"
Jul 21 15:06:06.153036 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.153004    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qdd9p_8407a2ae-c501-42e8-ab18-68e02523212a/kube-rbac-proxy/0.log"
Jul 21 15:06:06.189991 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.189965    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qdd9p_8407a2ae-c501-42e8-ab18-68e02523212a/init-textfile/0.log"
Jul 21 15:06:06.434973 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.434903    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/prometheus/0.log"
Jul 21 15:06:06.465003 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.464975    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/config-reloader/0.log"
Jul 21 15:06:06.497700 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.497670    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/thanos-sidecar/0.log"
Jul 21 15:06:06.530064 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.530037    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/kube-rbac-proxy-web/0.log"
Jul 21 15:06:06.560191 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.560160    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/kube-rbac-proxy/0.log"
Jul 21 15:06:06.586659 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.586628    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/kube-rbac-proxy-thanos/0.log"
Jul 21 15:06:06.623493 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:06.623462    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5ef10bfd-fd30-4abc-b014-702778f7470c/init-config-reloader/0.log"
Jul 21 15:06:08.036383 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.036341    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"]
Jul 21 15:06:08.047059 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.047016    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.057957 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.057889    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"]
Jul 21 15:06:08.167938 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.167901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-lib-modules\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.168114 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.167962    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-podres\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.168114 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.167988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-sys\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.168114 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.168103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-proc\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.168239 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.168149    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fnj8\" (UniqueName: \"kubernetes.io/projected/897921a8-8478-48dc-8cfb-b418c84943d3-kube-api-access-4fnj8\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.268710 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-podres\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.268922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-sys\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.268922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-proc\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.268922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268855    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fnj8\" (UniqueName: \"kubernetes.io/projected/897921a8-8478-48dc-8cfb-b418c84943d3-kube-api-access-4fnj8\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.268922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-lib-modules\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.268922 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268913    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-sys\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.269210 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.268991    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-proc\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.269210 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.269092    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-lib-modules\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.269210 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.269164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/897921a8-8478-48dc-8cfb-b418c84943d3-podres\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.280971 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.280934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fnj8\" (UniqueName: \"kubernetes.io/projected/897921a8-8478-48dc-8cfb-b418c84943d3-kube-api-access-4fnj8\") pod \"perf-node-gather-daemonset-kj2k8\" (UID: \"897921a8-8478-48dc-8cfb-b418c84943d3\") " pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.360103 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.360027    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.534565 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.534538    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"]
Jul 21 15:06:08.537787 ip-10-0-129-87 kubenswrapper[2569]: W0721 15:06:08.537757    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod897921a8_8478_48dc_8cfb_b418c84943d3.slice/crio-4408d2971d15a32b92a48d7077e13122f947bb8eba1cf5e340e6d53bd5bf5e47 WatchSource:0}: Error finding container 4408d2971d15a32b92a48d7077e13122f947bb8eba1cf5e340e6d53bd5bf5e47: Status 404 returned error can't find the container with id 4408d2971d15a32b92a48d7077e13122f947bb8eba1cf5e340e6d53bd5bf5e47
Jul 21 15:06:08.703775 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.703737    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8" event={"ID":"897921a8-8478-48dc-8cfb-b418c84943d3","Type":"ContainerStarted","Data":"76d585e13b6a04ee98042a9727035b22c6e27887aa21e1cdf1a6826005d76e6d"}
Jul 21 15:06:08.703937 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.703805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8" event={"ID":"897921a8-8478-48dc-8cfb-b418c84943d3","Type":"ContainerStarted","Data":"4408d2971d15a32b92a48d7077e13122f947bb8eba1cf5e340e6d53bd5bf5e47"}
Jul 21 15:06:08.703937 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.703835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:08.725232 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:08.725167    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8" podStartSLOduration=0.725146168 podStartE2EDuration="725.146168ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 15:06:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:06:08.724140161 +0000 UTC m=+2894.858535537" watchObservedRunningTime="2026-07-21 15:06:08.725146168 +0000 UTC m=+2894.859541542"
Jul 21 15:06:09.305867 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:09.305842    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-cc6bdcc65-vw7rz_fe1609b1-aa49-493c-90e3-eff174844f8b/console/0.log"
Jul 21 15:06:11.312779 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:11.312751    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4ftr_9918107e-39c4-411c-99f2-c7ebe307cb65/dns/0.log"
Jul 21 15:06:11.343905 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:11.343872    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4ftr_9918107e-39c4-411c-99f2-c7ebe307cb65/kube-rbac-proxy/0.log"
Jul 21 15:06:11.475395 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:11.475364    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n99kh_0e335d36-8172-432c-965c-7176028cf41f/dns-node-resolver/0.log"
Jul 21 15:06:12.089900 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:12.089872    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5888557764-m6s9h_062b0576-1508-4eed-b37b-cb1f0b1ae2b1/registry/0.log"
Jul 21 15:06:12.120970 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:12.120918    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7bvzz_b5a651ff-94a7-45df-a9a9-b0266bf28223/node-ca/0.log"
Jul 21 15:06:13.423915 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:13.423881    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd655967-thj96_dc629773-d072-453c-86ef-b5843a33fb3f/kube-auth-proxy/0.log"
Jul 21 15:06:14.494721 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:14.494680    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zm5zn_85ce1fac-29e6-46c0-8aed-8a6f33357deb/serve-healthcheck-canary/0.log"
Jul 21 15:06:14.720328 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:14.720298    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5jhqw/perf-node-gather-daemonset-kj2k8"
Jul 21 15:06:15.095292 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:15.095259    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4plps_ec673038-4e58-4f33-bafe-8155ac3a4efc/kube-rbac-proxy/0.log"
Jul 21 15:06:15.124987 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:15.124956    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4plps_ec673038-4e58-4f33-bafe-8155ac3a4efc/exporter/0.log"
Jul 21 15:06:15.153478 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:15.153439    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4plps_ec673038-4e58-4f33-bafe-8155ac3a4efc/extractor/0.log"
Jul 21 15:06:18.229906 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:18.229876    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd655967-thj96_dc629773-d072-453c-86ef-b5843a33fb3f/kube-auth-proxy/0.log"
Jul 21 15:06:18.240638 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:18.240609    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wwjvz_40c27eae-2b98-4af8-bc7b-7ae6a83b4529/manager/0.log"
Jul 21 15:06:18.765592 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:18.765560    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bb54d755-wrxfj_8e662208-e323-4a0f-b7ad-8b70cc0c8818/manager/0.log"
Jul 21 15:06:18.791189 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:18.791162    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bb54d755-wrxfj_8e662208-e323-4a0f-b7ad-8b70cc0c8818/copy-manifests/0.log"
Jul 21 15:06:19.691823 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:19.691789    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd655967-thj96_dc629773-d072-453c-86ef-b5843a33fb3f/kube-auth-proxy/0.log"
Jul 21 15:06:20.194180 ip-10-0-129-87 kubenswrapper[2569]: I0721 15:06:20.194144    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6df87d547c-gzps8_4284e49c-0e0a-4a92-a17e-8c4d6f38245d/manager/0.log"