Jul 10 14:32:52.028638 ip-10-0-136-210 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 14:32:52.028655 ip-10-0-136-210 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 14:32:52.028665 ip-10-0-136-210 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 14:32:52.028954 ip-10-0-136-210 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 14:33:02.148190 ip-10-0-136-210 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 14:33:02.148205 ip-10-0-136-210 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2f739aee4ba24e46b702ecbe3773ce7a --
Jul 10 14:35:02.179159 ip-10-0-136-210 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 14:35:02.548623 ip-10-0-136-210 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 14:35:02.548623 ip-10-0-136-210 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 14:35:02.548623 ip-10-0-136-210 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 14:35:02.548623 ip-10-0-136-210 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 14:35:02.548623 ip-10-0-136-210 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 14:35:02.549783 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.549620    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 14:35:02.554533 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554516    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:35:02.554533 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554532    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554536    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554540    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554543    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554547    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554550    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554553    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554556    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554559    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554562    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554565    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554568    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554571    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554573    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554576    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554579    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554582    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554585    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554587    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554604    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:35:02.554619 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554607    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554611    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554614    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554617    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554620    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554624    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554627    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554631    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554634    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554637    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554639    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554642    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554645    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554648    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554650    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554653    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554656    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554659    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554661    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554664    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:35:02.555114 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554667    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554669    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554672    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554675    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554678    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554681    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554683    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554686    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554689    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554691    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554694    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554698    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554700    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554703    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554708    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554711    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554713    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554716    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554719    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554722    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:35:02.555643 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554725    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554728    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554730    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554733    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554736    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554739    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554741    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554744    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554749    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554753    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554756    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554759    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554762    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554766    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554770    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554773    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554776    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554779    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554782    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:35:02.556164 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554786    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554789    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554792    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554795    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554798    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554801    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554803    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554807    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554810    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554813    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554816    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554819    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554822    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554826    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554829    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554832    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554835    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554840    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.554844    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:35:02.556664 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556603    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556612    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556616    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556619    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556623    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556625    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556628    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556633    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556635    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556641    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556645    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556648    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556651    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556655    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556659    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556662    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556666    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556669    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556672    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:35:02.557141 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556675    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556678    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556681    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556684    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556687    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556690    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556693    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556696    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556698    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556701    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556704    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556706    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556709    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556712    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556715    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556719    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556723    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556726    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556729    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556732    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:35:02.557682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556734    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556738    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556740    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556744    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556747    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556750    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556752    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556755    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556758    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556761    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556764    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556766    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556769    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556772    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556774    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556778    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556782    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556785    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556787    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556790    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:35:02.558257 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556793    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556796    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556799    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556802    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556804    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556808    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556810    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556814    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556817    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556820    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556822    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556825    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556828    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556830    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556833    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556836    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556839    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556842    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556844    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556847    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:35:02.558806 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556850    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556853    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556855    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556858    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556860    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556864    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556867    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556870    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556873    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556875    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556878    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556881    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556884    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556887    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556890    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556892    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556895    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556898    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556901    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.556904    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:35:02.559315 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557331    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557336    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557340    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557344    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557347    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557350    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557353    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557356    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557367    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557371    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557374    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557378    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557381    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557384    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557386    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557389    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557392    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557395    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557398    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:35:02.559844 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557401    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557403    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557406    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557409    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557411    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557414    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557417    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557420    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557422    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557426    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557429    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557432    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557435    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557438    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557441    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557443    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557446    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557449    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557451    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:35:02.560320 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557455    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557458    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557460    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557469    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557472    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557475    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557477    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557480    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557483    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557486    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557490    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557494    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557497    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557499    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557502    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557505    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557507    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557510    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557512    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557515    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:35:02.560824 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557518    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557520    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557523    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557531    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557533    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557536    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557539    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557542    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557546    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557548    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557551    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557554    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557557    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557559    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557562    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557564    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557569    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557573    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557577    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557580    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:35:02.561366 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557583    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557586    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557603    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557607    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557610    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557613    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557616    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557619    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557622    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557625    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557628    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557630    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557633    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557636    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557639    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557643    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557646    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557649    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557652    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557655    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.557658    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:35:02.561900 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557725    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557732    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557739    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557744    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557748    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557752    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557756    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557760    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557764    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557768    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557772    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557775    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557778    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557781    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557785    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557788    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557791    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557794    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557799    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557802    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557805    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557809    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557813    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557817    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 14:35:02.562421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557820    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557824    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557828    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557832    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557835    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557838    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557842    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557845    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557850    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557853    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557856    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557859    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557862    2583 flags.go:64] FLAG: --enable-server="true"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557865    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557870    2583 flags.go:64] FLAG: --event-burst="100"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557873    2583 flags.go:64] FLAG: --event-qps="50"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557877    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557880    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557884    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557888    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557891    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557894    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557897    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557900    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557903    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 14:35:02.563065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557906    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557909    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557912    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557915    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557918    2583 flags.go:64] FLAG: --feature-gates=""
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557922    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557926    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557929    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557932    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557935    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557941    2583 flags.go:64] FLAG: --help="false"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557944    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-136-210.ec2.internal"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557947    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557950    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557953    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557957    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557961    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557964    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557967    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557970    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557973    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557976    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.557980    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558000    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 10 14:35:02.563762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558023    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558029    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558033    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558037    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558041    2583 flags.go:64] FLAG: --lock-file=""
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558045    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558049    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558053    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558060    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558064    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558068    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558071    2583 flags.go:64] FLAG: --logging-format="text"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558074    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558078    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558082    2583 flags.go:64] FLAG: --manifest-url=""
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558085    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558091    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558094    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558099    2583 flags.go:64] FLAG: --max-pods="110"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558103    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558106    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558110    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558113    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558116    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558120    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 14:35:02.564377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558123    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558138    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558142    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558145    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558148    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558151    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558157    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558162    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558168    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558171    2583 flags.go:64] FLAG: --port="10250"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558174    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558178    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d860f506da822077"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558181    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558185    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558188    2583 flags.go:64] FLAG: --register-node="true"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558191    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558195    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558198    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558201    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558204    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558208    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558212    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558215    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558218    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558221    2583 flags.go:64] FLAG: --runonce="false"
Jul 10 14:35:02.565036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558224    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558228    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558231    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558234    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558237    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558241    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558244    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558247    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558250    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558253    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558256    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558260    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558263    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558267    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558270    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558280    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558283    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558286    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558292    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558296    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558299    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558302    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558305    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558308    2583 flags.go:64] FLAG: --v="2"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558313    2583 flags.go:64] FLAG: --version="false"
Jul 10 14:35:02.565695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558317    2583 flags.go:64] FLAG: --vmodule=""
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558321    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.558325    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558485    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558490    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558494    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558497    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558500    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558503    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558505    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558508    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558513    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558517    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558521    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558524    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558527    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558530    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558533    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558536    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558538    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:35:02.566343 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558541    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558544    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558549    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558553    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558556    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558572    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558575    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558577    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558580    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558583    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558586    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558589    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558592    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558595    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558598    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558601    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558604    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558607    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558609    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558612    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:35:02.566868 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558615    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558618    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558621    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558624    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558627    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558629    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558632    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558635    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558638    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558641    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558643    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558646    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558651    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558654    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558658    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558661    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558666    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558670    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558673    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558676    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:35:02.567388 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558679    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558681    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558684    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558687    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558690    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558692    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558696    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558698    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558701    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558703    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558706    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558709    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558712    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558716    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558720    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558724    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558727    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558730    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558733    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:35:02.568016 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558737    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558740    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558743    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558746    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558749    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558753    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558756    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558761    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558764    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558767    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558770    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558773    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558776    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558778    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558781    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558784    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558786    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558790    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558792    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558795    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:35:02.568515 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558798    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558800    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.558803    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.559771    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.567327    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.567347    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.567369    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.567375    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567445    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567451    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567454    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567458    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567461    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567464    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567467    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:35:02.569043 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567470    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567473    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567476    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567478    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567481    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567484    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567487    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567490    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567493    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567495    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567498    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567501    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567504    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567507    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567509    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567512    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567515    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567518    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567520    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567523    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:35:02.569441 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567526    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567529    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567539    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567542    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567545    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567548    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567551    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567554    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567557    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567560    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567563    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567566    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567568    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567571    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567574    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567576    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567581    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567584    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567587    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:35:02.570056 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567606    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567611    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567614    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567617    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567619    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567622    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567625    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567628    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567631    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567634    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567637    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567640    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567642    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567645    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567648    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567653    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567665    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567669    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567673    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:35:02.570776 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567677    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567681    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567684    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567686    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567689    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567692    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567695    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567698    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567700    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567703    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567706    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567709    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567712    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567715    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567717    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567720    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567722    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567725    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567728    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567731    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:35:02.571297 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567734    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567736    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567739    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567741    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567744    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567747    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567749    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567752    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567755    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567758    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567767    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567770    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567773    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567776    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.567781    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 14:35:02.571958 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567947    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567952    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567955    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567958    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567961    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567965    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567968    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567971    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567974    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567977    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567980    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567983    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567986    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567989    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567992    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567994    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567997    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.567999    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568002    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568005    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:35:02.572380 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568008    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568010    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568013    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568016    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568018    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568022    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568025    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568028    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568037    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568040    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568043    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568046    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568049    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568051    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568054    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568058    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568061    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568064    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568067    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:35:02.572931 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568070    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568073    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568076    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568079    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568082    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568085    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568088    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568090    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568093    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568096    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568098    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568101    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568104    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568107    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568109    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568112    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568114    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568117    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568120    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568123    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:35:02.573436 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568125    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568128    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568137    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568140    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568142    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568145    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568148    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568151    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568154    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568157    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568160    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568164    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568168    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568171    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568174    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568176    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568179    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568181    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568184    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:35:02.573963 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568187    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568189    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568192    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568195    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568197    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568200    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568203    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568205    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568208    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568211    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568214    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568217    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568219    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568222    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568224    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568227    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568236    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568238    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568241    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568243    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:35:02.574464 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:02.568246    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:35:02.575046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.568252    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 14:35:02.575046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.568402    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 14:35:02.575046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.573448    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 14:35:02.575046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.574923    2583 server.go:1019] "Starting client certificate rotation"
Jul 10 14:35:02.575046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.575024    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 14:35:02.575187 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.575062    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 14:35:02.577632 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.577613    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 14:35:02.578843 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.578818    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 14:35:02.592054 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.592032    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 10 14:35:02.594211 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.594195    2583 log.go:25] "Validated CRI v1 image API"
Jul 10 14:35:02.595173 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.595146    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 14:35:02.599001 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.598981    2583 fs.go:135] Filesystem UUIDs: map[4163cd0b-4b37-4965-8e83-a2598a0788ad:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7aea0cc3-750e-41ff-9edb-1d0481213f67:/dev/nvme0n1p4]
Jul 10 14:35:02.599071 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.599001    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 14:35:02.604912 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.604780    2583 manager.go:216] Machine: {Timestamp:2026-07-10 14:35:02.603415606 +0000 UTC m=+0.309737068 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100246 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f23d77cda979c94d60bd14f53b1bc SystemUUID:ec2f23d7-7cda-979c-94d6-0bd14f53b1bc BootID:2f739aee-4ba2-4e46-b702-ecbe3773ce7a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:62:24:ed:49:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:62:24:ed:49:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:c3:de:fd:20:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 14:35:02.604912 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.604910    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 14:35:02.605022 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.604994    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 14:35:02.608417 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.608384    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 14:35:02.608565 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.608419    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-210.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 14:35:02.609900 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.609886    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 14:35:02.609900 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.609901    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 14:35:02.609985 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.609908    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 14:35:02.610570 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.610559    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 14:35:02.612684 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.612667    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 14:35:02.612817 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.612802    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 14:35:02.616022 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616009    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 14:35:02.616164 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616153    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 14:35:02.616196 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616185    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 14:35:02.616196 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616194    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 14:35:02.616257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616209    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 14:35:02.616257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616220    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 10 14:35:02.616257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616229    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 14:35:02.616748 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616729    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 14:35:02.616791 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.616768    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 14:35:02.620394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.620367    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 14:35:02.622537 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622521    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 14:35:02.622617 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622554    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 14:35:02.622728 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622717    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 14:35:02.622772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622733    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 14:35:02.622772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622740    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 14:35:02.622772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622746    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 14:35:02.622772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622755    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 14:35:02.622772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.622761    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 14:35:02.623379 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.623369    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 14:35:02.623379 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.623379    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 14:35:02.623450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.623386    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 14:35:02.623450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.623392    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 14:35:02.623450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.623401    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 14:35:02.623450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.623409    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 14:35:02.624394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.624384    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 14:35:02.624394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.624394    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 14:35:02.627755 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.627739    2583 server.go:1268] "Started kubelet"
Jul 10 14:35:02.627884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.627858    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 14:35:02.627950 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.627906    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 14:35:02.628000 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.627967    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 14:35:02.628736 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.628716    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-210.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 14:35:02.628829 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.628749    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 14:35:02.628887 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.628869    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-210.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 14:35:02.629189 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.629168    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 14:35:02.629374 ip-10-0-136-210 systemd[1]: Started Kubernetes Kubelet.
Jul 10 14:35:02.630630 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.630611    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 14:35:02.635013 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.634988    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 14:35:02.636332 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.636310    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 14:35:02.636332 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.636317    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 14:35:02.636523 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.636403    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:02.637003 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.636981    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 14:35:02.637098 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637007    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 14:35:02.637098 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.636983    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 14:35:02.637231 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637129    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 14:35:02.637231 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637138    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 14:35:02.637328 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637266    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 14:35:02.637328 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637281    2583 factory.go:55] Registering systemd factory
Jul 10 14:35:02.637328 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637291    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 10 14:35:02.637670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637645    2583 factory.go:153] Registering CRI-O factory
Jul 10 14:35:02.637670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637664    2583 factory.go:223] Registration of the crio container factory successfully
Jul 10 14:35:02.637813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637688    2583 factory.go:103] Registering Raw factory
Jul 10 14:35:02.637813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.637701    2583 manager.go:1196] Started watching for new ooms in manager
Jul 10 14:35:02.638085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.638060    2583 manager.go:349] Starting recovery of all containers
Jul 10 14:35:02.638193 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.637180    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-210.ec2.internal.18c0f455ff157b03  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-210.ec2.internal,UID:ip-10-0-136-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-210.ec2.internal,},FirstTimestamp:2026-07-10 14:35:02.627711747 +0000 UTC m=+0.334033209,LastTimestamp:2026-07-10 14:35:02.627711747 +0000 UTC m=+0.334033209,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-210.ec2.internal,}"
Jul 10 14:35:02.649839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.649819    2583 manager.go:354] Recovery completed
Jul 10 14:35:02.649985 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.649958    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-210.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 14:35:02.650065 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.650016    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 14:35:02.654876 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.654863    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.655190 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.655173    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-85b84"
Jul 10 14:35:02.656762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.656711    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.656762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.656743    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.656762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.656757    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.657350 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.657334    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 14:35:02.657350 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.657347    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 14:35:02.657449 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.657366    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 14:35:02.657449 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.657377    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.658213 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.658190    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.658290 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.658219    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.658290 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.658230    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.658731 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.658714    2583 policy_none.go:49] "None policy: Start"
Jul 10 14:35:02.658731 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.658732    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 14:35:02.658818 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.658747    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 14:35:02.659850 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.659840    2583 policy_none.go:47] "Start"
Jul 10 14:35:02.665300 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.665281    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-85b84"
Jul 10 14:35:02.668705 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.668524    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-210.ec2.internal.18c0f45600d02f63  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-210.ec2.internal,UID:ip-10-0-136-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-210.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-210.ec2.internal,},FirstTimestamp:2026-07-10 14:35:02.656724835 +0000 UTC m=+0.363046298,LastTimestamp:2026-07-10 14:35:02.656724835 +0000 UTC m=+0.363046298,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-210.ec2.internal,}"
Jul 10 14:35:02.698226 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698208    2583 manager.go:337] "Starting Device Plugin manager"
Jul 10 14:35:02.698298 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.698240    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 14:35:02.698298 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698251    2583 server.go:87] "Starting device plugin registration server"
Jul 10 14:35:02.698525 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698511    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 14:35:02.698624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698528    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 14:35:02.698726 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698710    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 14:35:02.698804 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698793    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 14:35:02.698804 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.698803    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 14:35:02.699431 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.699401    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 14:35:02.699519 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.699451    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:02.800364 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.800304    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.801481 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.801206    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.801481 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.801235    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.801481 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.801248    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.801481 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.801273    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-210.ec2.internal"
Jul 10 14:35:02.813257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.813239    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-210.ec2.internal"
Jul 10 14:35:02.813346 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.813262    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-210.ec2.internal\": node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:02.827829 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.827803    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:02.838158 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.838124    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 14:35:02.839471 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.839446    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 14:35:02.839570 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.839490    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 14:35:02.839570 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.839514    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 14:35:02.839674 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.839580    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 14:35:02.843284 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.843263    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:35:02.929759 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.929721    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:02.940541 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.940506    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"]
Jul 10 14:35:02.940617 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.940583    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.942271 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.942253    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.942371 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.942283    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.942371 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.942295    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.943913 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.943900    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.944045 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944029    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:02.944126 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944062    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.944645 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944625    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.944711 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944655    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.944711 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944667    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.944784 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944626    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.944784 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944747    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.944784 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.944759    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.946054 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.946037    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:02.946148 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.946063    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:35:02.946780 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.946758    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:35:02.946857 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.946783    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:35:02.946857 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:02.946799    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:35:02.972321 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.972296    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-210.ec2.internal\" not found" node="ip-10-0-136-210.ec2.internal"
Jul 10 14:35:02.977000 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:02.976983    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-210.ec2.internal\" not found" node="ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.031898 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.031871    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:03.038895 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.038870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a95379fcdfcd10b1894525c55897d3ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal\" (UID: \"a95379fcdfcd10b1894525c55897d3ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.038979 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.038900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a95379fcdfcd10b1894525c55897d3ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal\" (UID: \"a95379fcdfcd10b1894525c55897d3ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.038979 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.038918    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d95b182bc9232c53452fc18d92789eb-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.ec2.internal\" (UID: \"3d95b182bc9232c53452fc18d92789eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.132416 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.132357    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:03.140014 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.139993    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a95379fcdfcd10b1894525c55897d3ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal\" (UID: \"a95379fcdfcd10b1894525c55897d3ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.140070 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.140021    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d95b182bc9232c53452fc18d92789eb-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.ec2.internal\" (UID: \"3d95b182bc9232c53452fc18d92789eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.140070 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.140046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a95379fcdfcd10b1894525c55897d3ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal\" (UID: \"a95379fcdfcd10b1894525c55897d3ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.140144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.140099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a95379fcdfcd10b1894525c55897d3ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal\" (UID: \"a95379fcdfcd10b1894525c55897d3ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.140144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.140124    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d95b182bc9232c53452fc18d92789eb-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.ec2.internal\" (UID: \"3d95b182bc9232c53452fc18d92789eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.140144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.140103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a95379fcdfcd10b1894525c55897d3ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal\" (UID: \"a95379fcdfcd10b1894525c55897d3ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.233644 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.233616    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:03.276360 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.276339    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.280888 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.280656    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.335185 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.335156    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:03.442848 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.442780    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-210.ec2.internal\" not found"
Jul 10 14:35:03.530610 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.530584    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:35:03.537216 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.537195    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.548899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.548879    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 14:35:03.550697 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.550683    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal"
Jul 10 14:35:03.568241 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.568223    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 14:35:03.574892 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.574876    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 14:35:03.575010 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.574993    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 14:35:03.575068 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.575049    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 14:35:03.617819 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.617796    2583 apiserver.go:52] "Watching apiserver"
Jul 10 14:35:03.636924 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.636904    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 14:35:03.646291 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.646267    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:35:03.649126 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.649098    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 14:35:03.649533 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.649493    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-dqlw9","openshift-network-diagnostics/network-check-target-b58rl","openshift-image-registry/node-ca-474nb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal","openshift-multus/multus-fg4tl","openshift-multus/network-metrics-daemon-dwdtq","openshift-network-operator/iptables-alerter-62n4t","openshift-ovn-kubernetes/ovnkube-node-s8565","kube-system/global-pull-secret-syncer-clcb4","kube-system/konnectivity-agent-25t26","kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl","openshift-cluster-node-tuning-operator/tuned-782ft","openshift-dns/node-resolver-prtg5"]
Jul 10 14:35:03.651711 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.651690    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.653034 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.653016    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:03.653116 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.653085    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:03.654306 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.654292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.655615 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.655581    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.656616 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.656596    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.656714 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.656618    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 14:35:03.657232 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.657211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:03.657316 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.657296    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:03.657518 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.657505    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 14:35:03.657742 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.657730    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 14:35:03.658316 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.658302    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.658806 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.658794    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.659033 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.659017    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-smk2c\""
Jul 10 14:35:03.660482 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.660464    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.661837 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.661822    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.661903 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.661872    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:03.662359 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.662337    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.662435 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.662342    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 14:35:03.663161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.663148    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.664213 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.664192    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.664757 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.664738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.666202 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.666173    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 14:35:03.666304 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.666261    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.667454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.667436    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.667791 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.667750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.667926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.667899    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.668143 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.668123    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.668682 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.668651    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 14:30:03 +0000 UTC" deadline="2028-01-30 09:12:52.713098103 +0000 UTC"
Jul 10 14:35:03.668759 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.668684    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13650h37m49.044418778s"
Jul 10 14:35:03.668759 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.668686    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 14:35:03.675711 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.675695    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 14:35:03.676520 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.676503    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 14:35:03.676679 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.676664    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vz2hb\""
Jul 10 14:35:03.676926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.676912    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8vntx\""
Jul 10 14:35:03.676964 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.676917    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mqn5s\""
Jul 10 14:35:03.677427 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.677413    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 14:35:03.677717 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.677695    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fftxb\""
Jul 10 14:35:03.677818 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.677695    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.677818 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.677730    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wbbdw\""
Jul 10 14:35:03.677818 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.677696    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 14:35:03.683749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.683733    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.684024 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684002    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.684115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684040    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 14:35:03.684115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684082    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.684115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684091    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 14:35:03.684239 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684053    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 14:35:03.684356 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684343    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.684449 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684436    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 14:35:03.684487 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.684442    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 14:35:03.689003 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.688989    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h4rzv\""
Jul 10 14:35:03.689088 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.689069    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qj582\""
Jul 10 14:35:03.689088 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.689078    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fkxmq\""
Jul 10 14:35:03.691595 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.689720    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 14:35:03.738463 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.738443    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 14:35:03.746515 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746500    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-systemd\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.746636 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-cni-binary-copy\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.746636 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746559    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-cni-multus\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.746636 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-etc-kubernetes\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.746636 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746597    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-ovn\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg2p5\" (UniqueName: \"kubernetes.io/projected/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-kube-api-access-sg2p5\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746746    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746763    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-device-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysctl-d\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.746813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746801    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-systemd-units\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746817    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-etc-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovnkube-script-lib\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.746992    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts8ct\" (UniqueName: \"kubernetes.io/projected/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-kube-api-access-ts8ct\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-kubelet\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747052    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-conf-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747080    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-cni-bin\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovn-node-metrics-cert\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747154    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-k8s-cni-cncf-io\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-registration-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747224    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5944324d-e133-48a0-920c-26ae31508a68-tmp-dir\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747250    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-kubelet-config\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747279    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-system-cni-dir\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747307    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-modprobe-d\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-cni-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-kubelet\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747377    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-log-socket\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747408    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-sys-fs\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.747440 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747435    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-os-release\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747463    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747489    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-host\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwkpc\" (UniqueName: \"kubernetes.io/projected/a887d595-6f78-4991-9532-c46e7bf126b4-kube-api-access-fwkpc\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-env-overrides\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-socket-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747616    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxq6\" (UniqueName: \"kubernetes.io/projected/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-kube-api-access-cvxq6\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747638    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dtgp\" (UniqueName: \"kubernetes.io/projected/5fdbffb2-cc33-4475-b081-9ab070bcb71b-kube-api-access-6dtgp\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747660    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-system-cni-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-var-lib-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747700    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cnibin\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdbffb2-cc33-4475-b081-9ab070bcb71b-host-slash\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747766    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysctl-conf\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747781    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-slash\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.747883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-kubernetes\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-var-lib-kubelet\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-cnibin\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-daemon-config\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747951    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gw8f\" (UniqueName: \"kubernetes.io/projected/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-kube-api-access-2gw8f\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.747972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-run\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748021    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-os-release\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-socket-dir-parent\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-sys\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5944324d-e133-48a0-920c-26ae31508a68-hosts-file\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748098    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-multus-certs\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-run-netns\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748134    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-node-log\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovnkube-config\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9528\" (UniqueName: \"kubernetes.io/projected/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-kube-api-access-r9528\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.748454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3055e9b1-3af7-4d56-83ca-574b55464161-serviceca\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748292    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysconfig\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748324    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-cni-bin\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-systemd\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-dbus\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9c33d692-b328-4356-982f-03e784e200a3-agent-certs\") pod \"konnectivity-agent-25t26\" (UID: \"9c33d692-b328-4356-982f-03e784e200a3\") " pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9c33d692-b328-4356-982f-03e784e200a3-konnectivity-ca\") pod \"konnectivity-agent-25t26\" (UID: \"9c33d692-b328-4356-982f-03e784e200a3\") " pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-lib-modules\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748506    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-tuned\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748597    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3055e9b1-3af7-4d56-83ca-574b55464161-host\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748630    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fdbffb2-cc33-4475-b081-9ab070bcb71b-iptables-alerter-script\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-tmp\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhg69\" (UniqueName: \"kubernetes.io/projected/5944324d-e133-48a0-920c-26ae31508a68-kube-api-access-vhg69\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.749085 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-netns\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.749621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-hostroot\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.749621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-cni-netd\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.749621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.748773    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24tf\" (UniqueName: \"kubernetes.io/projected/3055e9b1-3af7-4d56-83ca-574b55464161-kube-api-access-l24tf\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.763362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.763343    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:35:03.780702 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.780671    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jkh7l"
Jul 10 14:35:03.809423 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.809402    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jkh7l"
Jul 10 14:35:03.849935 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.849914    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-log-socket\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850028 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.849943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-sys-fs\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.850028 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.849961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-os-release\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850045    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-sys-fs\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.850104 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850078    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-log-socket\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850172 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850172 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-host\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.850172 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwkpc\" (UniqueName: \"kubernetes.io/projected/a887d595-6f78-4991-9532-c46e7bf126b4-kube-api-access-fwkpc\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:03.850172 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-env-overrides\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-os-release\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850207    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-host\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-socket-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850265    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxq6\" (UniqueName: \"kubernetes.io/projected/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-kube-api-access-cvxq6\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dtgp\" (UniqueName: \"kubernetes.io/projected/5fdbffb2-cc33-4475-b081-9ab070bcb71b-kube-api-access-6dtgp\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-system-cni-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.850373 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-var-lib-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850395    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-socket-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cnibin\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850460    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-system-cni-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850475    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-var-lib-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850516    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cnibin\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdbffb2-cc33-4475-b081-9ab070bcb71b-host-slash\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysctl-conf\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-slash\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850662    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdbffb2-cc33-4475-b081-9ab070bcb71b-host-slash\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850673    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-env-overrides\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850682    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850730    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-slash\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.850732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-kubernetes\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysctl-conf\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850775    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-kubernetes\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-var-lib-kubelet\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-cnibin\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-var-lib-kubelet\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850845    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-daemon-config\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gw8f\" (UniqueName: \"kubernetes.io/projected/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-kube-api-access-2gw8f\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850878    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-cnibin\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-run\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-os-release\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850976    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-socket-dir-parent\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.850984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-run\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-sys\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5944324d-e133-48a0-920c-26ae31508a68-hosts-file\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-socket-dir-parent\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.851422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-multus-certs\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-os-release\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851081    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-run-netns\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.851055    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5944324d-e133-48a0-920c-26ae31508a68-hosts-file\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-run-netns\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-multus-certs\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-node-log\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-sys\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851142    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.851166    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:04.351144877 +0000 UTC m=+2.057466344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-node-log\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851197    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovnkube-config\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9528\" (UniqueName: \"kubernetes.io/projected/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-kube-api-access-r9528\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3055e9b1-3af7-4d56-83ca-574b55464161-serviceca\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851291    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysconfig\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.852317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-cni-bin\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851340    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-systemd\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-cni-bin\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851367    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-dbus\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851420    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9c33d692-b328-4356-982f-03e784e200a3-agent-certs\") pod \"konnectivity-agent-25t26\" (UID: \"9c33d692-b328-4356-982f-03e784e200a3\") " pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9c33d692-b328-4356-982f-03e784e200a3-konnectivity-ca\") pod \"konnectivity-agent-25t26\" (UID: \"9c33d692-b328-4356-982f-03e784e200a3\") " pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851449    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-lib-modules\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-tuned\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851526    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-daemon-config\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3055e9b1-3af7-4d56-83ca-574b55464161-host\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fdbffb2-cc33-4475-b081-9ab070bcb71b-iptables-alerter-script\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851641    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-tmp\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.853279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhg69\" (UniqueName: \"kubernetes.io/projected/5944324d-e133-48a0-920c-26ae31508a68-kube-api-access-vhg69\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-netns\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851706    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovnkube-config\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851758    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3055e9b1-3af7-4d56-83ca-574b55464161-host\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851769    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-hostroot\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-hostroot\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851804    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851818    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-cni-netd\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851853    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-cni-netd\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24tf\" (UniqueName: \"kubernetes.io/projected/3055e9b1-3af7-4d56-83ca-574b55464161-kube-api-access-l24tf\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-systemd\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-lib-modules\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-cni-binary-copy\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851945    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-cni-multus\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-etc-kubernetes\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852012    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3055e9b1-3af7-4d56-83ca-574b55464161-serviceca\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.854188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-ovn\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852059    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-systemd\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg2p5\" (UniqueName: \"kubernetes.io/projected/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-kube-api-access-sg2p5\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9c33d692-b328-4356-982f-03e784e200a3-konnectivity-ca\") pod \"konnectivity-agent-25t26\" (UID: \"9c33d692-b328-4356-982f-03e784e200a3\") " pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-netns\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.851373    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysconfig\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-cni-multus\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852183    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-dbus\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.852206    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-device-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852290    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-systemd\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.852305    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:35:04.35228821 +0000 UTC m=+2.058609674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852313    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fdbffb2-cc33-4475-b081-9ab070bcb71b-iptables-alerter-script\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852332    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852337    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysctl-d\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.855130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852342    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-device-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852377    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-systemd-units\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852380    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-ovn\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-etc-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852417    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-etc-kubernetes\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-etc-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-systemd-units\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852514    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-sysctl-d\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovnkube-script-lib\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852581    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-run-openvswitch\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts8ct\" (UniqueName: \"kubernetes.io/projected/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-kube-api-access-ts8ct\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852723    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-cni-binary-copy\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852771    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-kubelet\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852804    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-conf-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-cni-bin\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852863    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovn-node-metrics-cert\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.855951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-k8s-cni-cncf-io\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-registration-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5944324d-e133-48a0-920c-26ae31508a68-tmp-dir\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852947    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-kubelet\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852917    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-conf-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852968    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-kubelet-config\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.852996    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-run-k8s-cni-cncf-io\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-registration-dir\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853033    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-kubelet-config\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853084    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovnkube-script-lib\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853145    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-host-cni-bin\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853175    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-system-cni-dir\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853204    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-modprobe-d\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853214    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-system-cni-dir\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853231    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-cni-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-kubelet\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.856491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5944324d-e133-48a0-920c-26ae31508a68-tmp-dir\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853328    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-host-var-lib-kubelet\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853312    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-modprobe-d\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.853367    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-multus-cni-dir\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.855579    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-tmp\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.855690    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-etc-tuned\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.855856    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-ovn-node-metrics-cert\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.857006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.855875    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9c33d692-b328-4356-982f-03e784e200a3-agent-certs\") pod \"konnectivity-agent-25t26\" (UID: \"9c33d692-b328-4356-982f-03e784e200a3\") " pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:03.859079 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:03.859057    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95379fcdfcd10b1894525c55897d3ca.slice/crio-635955b4fcba4539920ba0734dffedfe7032b53f4f356969158238a2c0daa6f8 WatchSource:0}: Error finding container 635955b4fcba4539920ba0734dffedfe7032b53f4f356969158238a2c0daa6f8: Status 404 returned error can't find the container with id 635955b4fcba4539920ba0734dffedfe7032b53f4f356969158238a2c0daa6f8
Jul 10 14:35:03.863170 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.863155    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:35:03.867764 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.867745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwkpc\" (UniqueName: \"kubernetes.io/projected/a887d595-6f78-4991-9532-c46e7bf126b4-kube-api-access-fwkpc\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:03.873891 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.873872    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:03.873994 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.873905    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:03.873994 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.873916    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:03.873994 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:03.873965    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:04.373953263 +0000 UTC m=+2.080274734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:03.875529 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.875494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhg69\" (UniqueName: \"kubernetes.io/projected/5944324d-e133-48a0-920c-26ae31508a68-kube-api-access-vhg69\") pod \"node-resolver-prtg5\" (UID: \"5944324d-e133-48a0-920c-26ae31508a68\") " pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:03.876170 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.876149    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dtgp\" (UniqueName: \"kubernetes.io/projected/5fdbffb2-cc33-4475-b081-9ab070bcb71b-kube-api-access-6dtgp\") pod \"iptables-alerter-62n4t\" (UID: \"5fdbffb2-cc33-4475-b081-9ab070bcb71b\") " pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:03.878855 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.878836    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24tf\" (UniqueName: \"kubernetes.io/projected/3055e9b1-3af7-4d56-83ca-574b55464161-kube-api-access-l24tf\") pod \"node-ca-474nb\" (UID: \"3055e9b1-3af7-4d56-83ca-574b55464161\") " pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.879465 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.879449    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts8ct\" (UniqueName: \"kubernetes.io/projected/8d379cfa-af47-47ab-8d38-5fb4c2ea46ea-kube-api-access-ts8ct\") pod \"tuned-782ft\" (UID: \"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea\") " pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:03.881222 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.881204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gw8f\" (UniqueName: \"kubernetes.io/projected/9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0-kube-api-access-2gw8f\") pod \"multus-fg4tl\" (UID: \"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0\") " pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:03.884097 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.884076    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9528\" (UniqueName: \"kubernetes.io/projected/8e7f9fd7-3204-4dd0-a791-a7ac41c068c1-kube-api-access-r9528\") pod \"aws-ebs-csi-driver-node-v8dpl\" (UID: \"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:03.884343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.884322    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg2p5\" (UniqueName: \"kubernetes.io/projected/43fa7fd7-d317-44b9-91b8-23fb1f464ad2-kube-api-access-sg2p5\") pod \"ovnkube-node-s8565\" (UID: \"43fa7fd7-d317-44b9-91b8-23fb1f464ad2\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:03.884537 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.884518    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxq6\" (UniqueName: \"kubernetes.io/projected/44846e01-1b2b-453b-ab4a-68fd1b2a28e7-kube-api-access-cvxq6\") pod \"multus-additional-cni-plugins-dqlw9\" (UID: \"44846e01-1b2b-453b-ab4a-68fd1b2a28e7\") " pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.975490 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.975416    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dqlw9"
Jul 10 14:35:03.979996 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:03.979978    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-474nb"
Jul 10 14:35:03.988208 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:03.988183    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44846e01_1b2b_453b_ab4a_68fd1b2a28e7.slice/crio-3f605c69aefcc99c7c0a105095577dcf31af14842aeaac6453acedef6ee182fd WatchSource:0}: Error finding container 3f605c69aefcc99c7c0a105095577dcf31af14842aeaac6453acedef6ee182fd: Status 404 returned error can't find the container with id 3f605c69aefcc99c7c0a105095577dcf31af14842aeaac6453acedef6ee182fd
Jul 10 14:35:03.991180 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:03.991157    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3055e9b1_3af7_4d56_83ca_574b55464161.slice/crio-864a0f9a84897bdba00e644c65b705f8d2c723c2341b2a6c8c62af0f27ac3a30 WatchSource:0}: Error finding container 864a0f9a84897bdba00e644c65b705f8d2c723c2341b2a6c8c62af0f27ac3a30: Status 404 returned error can't find the container with id 864a0f9a84897bdba00e644c65b705f8d2c723c2341b2a6c8c62af0f27ac3a30
Jul 10 14:35:04.010369 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.010345    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fg4tl"
Jul 10 14:35:04.018024 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.018000    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4e2aee_9629_48bc_b6fb_c4e6bc4adeb0.slice/crio-4ccb6056a2724d9e53f39b8d7cb07d6549e41158bdc1b3d5af4e7d50b14aef7d WatchSource:0}: Error finding container 4ccb6056a2724d9e53f39b8d7cb07d6549e41158bdc1b3d5af4e7d50b14aef7d: Status 404 returned error can't find the container with id 4ccb6056a2724d9e53f39b8d7cb07d6549e41158bdc1b3d5af4e7d50b14aef7d
Jul 10 14:35:04.024115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.024095    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-62n4t"
Jul 10 14:35:04.031247 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.031221    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdbffb2_cc33_4475_b081_9ab070bcb71b.slice/crio-2441531d0d8a550d67abc1316f15490669b918d23a468a06387bc0171a20ccb8 WatchSource:0}: Error finding container 2441531d0d8a550d67abc1316f15490669b918d23a468a06387bc0171a20ccb8: Status 404 returned error can't find the container with id 2441531d0d8a550d67abc1316f15490669b918d23a468a06387bc0171a20ccb8
Jul 10 14:35:04.043284 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.043264    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:04.049979 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.049956    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fa7fd7_d317_44b9_91b8_23fb1f464ad2.slice/crio-2e69b471940713dcdce14f50131cc68935d98e14f7733f54b04ad4707d77c3fb WatchSource:0}: Error finding container 2e69b471940713dcdce14f50131cc68935d98e14f7733f54b04ad4707d77c3fb: Status 404 returned error can't find the container with id 2e69b471940713dcdce14f50131cc68935d98e14f7733f54b04ad4707d77c3fb
Jul 10 14:35:04.063631 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.063612    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:04.071273 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.071249    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c33d692_b328_4356_982f_03e784e200a3.slice/crio-0c5811521f9e49127df7bbffdf97d396c5cdadba6f88fa2fb9ba913cbb362795 WatchSource:0}: Error finding container 0c5811521f9e49127df7bbffdf97d396c5cdadba6f88fa2fb9ba913cbb362795: Status 404 returned error can't find the container with id 0c5811521f9e49127df7bbffdf97d396c5cdadba6f88fa2fb9ba913cbb362795
Jul 10 14:35:04.081691 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.081673    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl"
Jul 10 14:35:04.089865 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.089838    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7f9fd7_3204_4dd0_a791_a7ac41c068c1.slice/crio-a41d6132a06fdef49bd2b526f07b4d28973e8dac42035fa253739671d8556d8a WatchSource:0}: Error finding container a41d6132a06fdef49bd2b526f07b4d28973e8dac42035fa253739671d8556d8a: Status 404 returned error can't find the container with id a41d6132a06fdef49bd2b526f07b4d28973e8dac42035fa253739671d8556d8a
Jul 10 14:35:04.094463 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.094443    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-782ft"
Jul 10 14:35:04.100554 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.100531    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-prtg5"
Jul 10 14:35:04.101100 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.101080    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d379cfa_af47_47ab_8d38_5fb4c2ea46ea.slice/crio-69a6db7202b5e0b5ae00ae3e5405cbb67de9d1e647e01d6063a3cdff5f6eebcf WatchSource:0}: Error finding container 69a6db7202b5e0b5ae00ae3e5405cbb67de9d1e647e01d6063a3cdff5f6eebcf: Status 404 returned error can't find the container with id 69a6db7202b5e0b5ae00ae3e5405cbb67de9d1e647e01d6063a3cdff5f6eebcf
Jul 10 14:35:04.109204 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.109184    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5944324d_e133_48a0_920c_26ae31508a68.slice/crio-8f5ee2f1f1d037850a53a0233b72255d57bd348a1f5d556b81369c8c8c2fa441 WatchSource:0}: Error finding container 8f5ee2f1f1d037850a53a0233b72255d57bd348a1f5d556b81369c8c8c2fa441: Status 404 returned error can't find the container with id 8f5ee2f1f1d037850a53a0233b72255d57bd348a1f5d556b81369c8c8c2fa441
Jul 10 14:35:04.149583 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:04.149562    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d95b182bc9232c53452fc18d92789eb.slice/crio-e056009b2ee0ee86daf21f5f10a76645d6077693473447068ab4ece7bbd3890c WatchSource:0}: Error finding container e056009b2ee0ee86daf21f5f10a76645d6077693473447068ab4ece7bbd3890c: Status 404 returned error can't find the container with id e056009b2ee0ee86daf21f5f10a76645d6077693473447068ab4ece7bbd3890c
Jul 10 14:35:04.356795 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.356688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:04.356795 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.356776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:04.356976 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.356841    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:04.356976 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.356877    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:04.356976 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.356907    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:35:05.356892095 +0000 UTC m=+3.063213547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:04.356976 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.356927    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:05.356913787 +0000 UTC m=+3.063235253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:04.457899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.457851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:04.458103 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.458084    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:04.458186 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.458111    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:04.458186 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.458125    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:04.458376 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.458189    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:05.458168458 +0000 UTC m=+3.164489925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:04.535645 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.535613    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:35:04.811541 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.811431    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:30:04 +0000 UTC" deadline="2028-03-08 13:16:50.794510315 +0000 UTC"
Jul 10 14:35:04.811541 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.811487    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14566h41m45.983027547s"
Jul 10 14:35:04.842485 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.842449    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:04.842664 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:04.842582    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:04.860533 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.860467    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal" event={"ID":"3d95b182bc9232c53452fc18d92789eb","Type":"ContainerStarted","Data":"e056009b2ee0ee86daf21f5f10a76645d6077693473447068ab4ece7bbd3890c"}
Jul 10 14:35:04.865997 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.865959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-782ft" event={"ID":"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea","Type":"ContainerStarted","Data":"69a6db7202b5e0b5ae00ae3e5405cbb67de9d1e647e01d6063a3cdff5f6eebcf"}
Jul 10 14:35:04.875443 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.873789    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"2e69b471940713dcdce14f50131cc68935d98e14f7733f54b04ad4707d77c3fb"}
Jul 10 14:35:04.884748 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.884689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fg4tl" event={"ID":"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0","Type":"ContainerStarted","Data":"4ccb6056a2724d9e53f39b8d7cb07d6549e41158bdc1b3d5af4e7d50b14aef7d"}
Jul 10 14:35:04.889621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.887708    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-474nb" event={"ID":"3055e9b1-3af7-4d56-83ca-574b55464161","Type":"ContainerStarted","Data":"864a0f9a84897bdba00e644c65b705f8d2c723c2341b2a6c8c62af0f27ac3a30"}
Jul 10 14:35:04.896770 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.896741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerStarted","Data":"3f605c69aefcc99c7c0a105095577dcf31af14842aeaac6453acedef6ee182fd"}
Jul 10 14:35:04.909230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.909192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-prtg5" event={"ID":"5944324d-e133-48a0-920c-26ae31508a68","Type":"ContainerStarted","Data":"8f5ee2f1f1d037850a53a0233b72255d57bd348a1f5d556b81369c8c8c2fa441"}
Jul 10 14:35:04.917623 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.917579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl" event={"ID":"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1","Type":"ContainerStarted","Data":"a41d6132a06fdef49bd2b526f07b4d28973e8dac42035fa253739671d8556d8a"}
Jul 10 14:35:04.921923 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.921570    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-25t26" event={"ID":"9c33d692-b328-4356-982f-03e784e200a3","Type":"ContainerStarted","Data":"0c5811521f9e49127df7bbffdf97d396c5cdadba6f88fa2fb9ba913cbb362795"}
Jul 10 14:35:04.931522 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.931495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-62n4t" event={"ID":"5fdbffb2-cc33-4475-b081-9ab070bcb71b","Type":"ContainerStarted","Data":"2441531d0d8a550d67abc1316f15490669b918d23a468a06387bc0171a20ccb8"}
Jul 10 14:35:04.939076 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:04.939046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal" event={"ID":"a95379fcdfcd10b1894525c55897d3ca","Type":"ContainerStarted","Data":"635955b4fcba4539920ba0734dffedfe7032b53f4f356969158238a2c0daa6f8"}
Jul 10 14:35:05.364907 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.364866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:05.365099 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.364966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:05.365176 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.365105    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:05.365176 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.365173    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:07.365151467 +0000 UTC m=+5.071472920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:05.365636 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.365612    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:05.365725 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.365677    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:35:07.36566026 +0000 UTC m=+5.071981715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:05.465622 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.465585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:05.465833 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.465812    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:05.465911 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.465837    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:05.465911 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.465850    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:05.465911 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.465909    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:07.465890034 +0000 UTC m=+5.172211518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:05.490543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.490514    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:35:05.811851 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.811725    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:30:04 +0000 UTC" deadline="2028-03-05 00:41:42.328505607 +0000 UTC"
Jul 10 14:35:05.811851 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.811795    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14482h6m36.516715758s"
Jul 10 14:35:05.840159 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.840127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:05.840343 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.840259    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:05.840732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:05.840708    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:05.840854 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:05.840832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:06.840800 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:06.840769    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:06.841261 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:06.840915    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:07.382176 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:07.382134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:07.382371 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:07.382207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:07.382371 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.382338    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:07.382507 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.382405    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:35:11.382383588 +0000 UTC m=+9.088705037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:07.382876 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.382849    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:07.382977 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.382914    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:11.382897379 +0000 UTC m=+9.089218838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:07.483265 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:07.483127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:07.483440 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.483420    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:07.483521 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.483445    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:07.483521 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.483461    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:07.483659 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.483523    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:11.483504437 +0000 UTC m=+9.189825907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:07.841045 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:07.840743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:07.841045 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:07.840809    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:07.841045 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.840911    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:07.841646 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:07.841055    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:08.841485 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:08.840965    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:08.841485 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:08.841115    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:09.839944 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:09.839911    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:09.840136 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:09.840037    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:09.840136 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:09.840103    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:09.840251 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:09.840214    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:10.840265 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:10.840132    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:10.840791 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:10.840279    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:11.419914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:11.419712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:11.419914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:11.419830    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:11.419914 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.419889    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:11.420170 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.419955    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:11.420170 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.419965    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:35:19.419943468 +0000 UTC m=+17.126264932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:11.420170 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.420008    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:19.419992652 +0000 UTC m=+17.126314105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:11.521115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:11.520454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:11.521115 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.520684    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:11.521115 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.520706    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:11.521115 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.520719    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:11.521115 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.520779    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:19.520761617 +0000 UTC m=+17.227083069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:11.839972 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:11.839886    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:11.840120 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.840030    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:11.840120 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:11.840105    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:11.840247 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:11.840218    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:12.841602 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:12.841568    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:12.842065 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:12.841689    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:13.839896 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:13.839863    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:13.840079 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:13.839903    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:13.840079 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:13.840002    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:13.840194 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:13.840130    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:14.840455 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:14.840417    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:14.840894 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:14.840552    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:15.840318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:15.840282    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:15.840494 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:15.840393    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:15.840494 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:15.840436    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:15.840829 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:15.840560    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:16.840804 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:16.840767    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:16.841263 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:16.840903    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:17.839774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:17.839726    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:17.839774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:17.839766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:17.840019 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:17.839883    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:17.840074 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:17.840009    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:18.840223 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:18.840180    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:18.840691 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:18.840316    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:19.480258 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:19.480220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:19.480258 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:19.480268    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:19.480632 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.480381    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:19.480632 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.480383    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:19.480632 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.480460    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:35:35.480430415 +0000 UTC m=+33.186751864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:19.480632 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.480476    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:35.480468835 +0000 UTC m=+33.186790281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:19.581472 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:19.581425    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:19.581643 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.581603    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:19.581643 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.581621    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:19.581643 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.581631    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:19.581774 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.581702    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:35.581685766 +0000 UTC m=+33.288007219 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:19.839799 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:19.839711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:19.839947 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:19.839721    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:19.839947 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.839841    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:19.839947 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:19.839914    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:20.839988 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:20.839813    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:20.840372 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:20.840092    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:21.839697 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:21.839666    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:21.839884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:21.839666    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:21.839884 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:21.839793    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:21.839976 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:21.839889    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:22.841414 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:22.841377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:22.841887 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:22.841500    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:23.840230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:23.840187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:23.840230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:23.840222    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:23.840480 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:23.840310    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:23.840480 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:23.840455    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:24.840877 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.840678    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:24.840877 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:24.840839    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:24.989125 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.988867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal" event={"ID":"3d95b182bc9232c53452fc18d92789eb","Type":"ContainerStarted","Data":"bcb8a9a04f693a548943a61613ce9d05a43fab90a59460ac2129c9a34eeffe36"}
Jul 10 14:35:24.990574 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.990498    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-782ft" event={"ID":"8d379cfa-af47-47ab-8d38-5fb4c2ea46ea","Type":"ContainerStarted","Data":"5987801a570db018c40df87f3d500ec4b0c086ffea40cc397f4b766406d65a22"}
Jul 10 14:35:24.995972 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.995924    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:35:24.996222 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.996201    2583 generic.go:356] "Generic (PLEG): container finished" podID="43fa7fd7-d317-44b9-91b8-23fb1f464ad2" containerID="203649f7d0b71a816c0acdb6eb579fd60173bf78868444af08ce0a1c7b9f8f29" exitCode=1
Jul 10 14:35:24.996310 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.996279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"1ced38ac36376d1c87227ac38f22cd3681a66c71b438ec964c6cb1e617eff156"}
Jul 10 14:35:24.996353 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.996345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerDied","Data":"203649f7d0b71a816c0acdb6eb579fd60173bf78868444af08ce0a1c7b9f8f29"}
Jul 10 14:35:24.996388 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.996376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"2dde616e406da7fd3e29247b5cefc4768dbf1a9bf73f6ab23606ac7c765b07f5"}
Jul 10 14:35:24.997390 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.997372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fg4tl" event={"ID":"9c4e2aee-9629-48bc-b6fb-c4e6bc4adeb0","Type":"ContainerStarted","Data":"5ff20ebf4df4bb253bea02a1d022318bde043f27707a61cc2770c21b96250d60"}
Jul 10 14:35:24.998756 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:24.998730    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-474nb" event={"ID":"3055e9b1-3af7-4d56-83ca-574b55464161","Type":"ContainerStarted","Data":"4e81d2c79acfd47a032ccf71c5ba3975d9a03926254b1987a41cf37d5e6babc1"}
Jul 10 14:35:25.000211 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.000192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerStarted","Data":"353d3d8d6e68a7e56a3dfd80a401be05aa4958d37a8bbd8bbad67d8efa2394cc"}
Jul 10 14:35:25.008264 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.008197    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-prtg5" event={"ID":"5944324d-e133-48a0-920c-26ae31508a68","Type":"ContainerStarted","Data":"cadf65fde9e86268504fb94080a714f6a39136f23e0c70af670b0bb126fd61c7"}
Jul 10 14:35:25.009693 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.009628    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl" event={"ID":"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1","Type":"ContainerStarted","Data":"93dcb916545e817026ce145837494cf2340f98a9b0073b3710183f11245620c0"}
Jul 10 14:35:25.011290 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.011031    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-25t26" event={"ID":"9c33d692-b328-4356-982f-03e784e200a3","Type":"ContainerStarted","Data":"74b81ef185ed5f0266b837d040c3b9ccc14adbbc2144b80673b2e831f481761f"}
Jul 10 14:35:25.012848 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.012825    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal" event={"ID":"a95379fcdfcd10b1894525c55897d3ca","Type":"ContainerStarted","Data":"de3690e0d2b8bff7ce96659586bf06e7e9b87b47b8123029c84253d0b2585362"}
Jul 10 14:35:25.021069 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.021022    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.ec2.internal" podStartSLOduration=22.020991754 podStartE2EDuration="22.020991754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:25.020694035 +0000 UTC m=+22.727015505" watchObservedRunningTime="2026-07-10 14:35:25.020991754 +0000 UTC m=+22.727313223"
Jul 10 14:35:25.132269 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.132048    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-474nb" podStartSLOduration=1.5884841889999999 podStartE2EDuration="22.132027439s" totalImagesPullingTime="20.54354325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:25.072240067 +0000 UTC m=+22.778561536" watchObservedRunningTime="2026-07-10 14:35:25.132027439 +0000 UTC m=+22.838348909"
Jul 10 14:35:25.215649 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.215599    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-25t26" podStartSLOduration=1.77284461 podStartE2EDuration="22.215583667s" totalImagesPullingTime="20.442739057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:25.167278319 +0000 UTC m=+22.873599785" watchObservedRunningTime="2026-07-10 14:35:25.215583667 +0000 UTC m=+22.921905136"
Jul 10 14:35:25.215878 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.215852    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-782ft" podStartSLOduration=1.779168173 podStartE2EDuration="22.215841844s" totalImagesPullingTime="20.436673671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:25.215719838 +0000 UTC m=+22.922041289" watchObservedRunningTime="2026-07-10 14:35:25.215841844 +0000 UTC m=+22.922163313"
Jul 10 14:35:25.282017 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.281982    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-prtg5" podStartSLOduration=1.9276743139999999 podStartE2EDuration="22.281968594s" totalImagesPullingTime="20.35429428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:25.271815284 +0000 UTC m=+22.978136752" watchObservedRunningTime="2026-07-10 14:35:25.281968594 +0000 UTC m=+22.988290062"
Jul 10 14:35:25.840675 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.840593    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:25.840879 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:25.840593    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:25.840879 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:25.840721    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:25.840879 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:25.840832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:26.016010 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.015974    2583 generic.go:356] "Generic (PLEG): container finished" podID="a95379fcdfcd10b1894525c55897d3ca" containerID="de3690e0d2b8bff7ce96659586bf06e7e9b87b47b8123029c84253d0b2585362" exitCode=0
Jul 10 14:35:26.016168 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.016063    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal" event={"ID":"a95379fcdfcd10b1894525c55897d3ca","Type":"ContainerDied","Data":"de3690e0d2b8bff7ce96659586bf06e7e9b87b47b8123029c84253d0b2585362"}
Jul 10 14:35:26.016168 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.016106    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal" event={"ID":"a95379fcdfcd10b1894525c55897d3ca","Type":"ContainerStarted","Data":"06bb0e10bd7e4b970bf519658ad26d2304209d19b8ee1abfb9c24c44844dc5ad"}
Jul 10 14:35:26.018697 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.018678    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:35:26.019051 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.019036    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"95d84534cd10ad4022a2446dab501300dd7504172de6da1429e3fc60af916d14"}
Jul 10 14:35:26.019131 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.019092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"847ccbc97e6c04924c7c6a1472e55c8beb80cab55224591ed93d54df1b1ede7c"}
Jul 10 14:35:26.019131 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.019121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"a84766177377e97e80439da8502cac47f6475d042e6249fe1a6d96aab491e57d"}
Jul 10 14:35:26.020315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.020297    2583 generic.go:356] "Generic (PLEG): container finished" podID="44846e01-1b2b-453b-ab4a-68fd1b2a28e7" containerID="353d3d8d6e68a7e56a3dfd80a401be05aa4958d37a8bbd8bbad67d8efa2394cc" exitCode=0
Jul 10 14:35:26.020407 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.020369    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerDied","Data":"353d3d8d6e68a7e56a3dfd80a401be05aa4958d37a8bbd8bbad67d8efa2394cc"}
Jul 10 14:35:26.034941 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.034897    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.ec2.internal" podStartSLOduration=23.034884904 podStartE2EDuration="23.034884904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:26.034582388 +0000 UTC m=+23.740903857" watchObservedRunningTime="2026-07-10 14:35:26.034884904 +0000 UTC m=+23.741206373"
Jul 10 14:35:26.035041 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.034967    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fg4tl" podStartSLOduration=2.328232095 podStartE2EDuration="23.03496294s" totalImagesPullingTime="20.706730845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:25.367199017 +0000 UTC m=+23.073520486" watchObservedRunningTime="2026-07-10 14:35:26.03496294 +0000 UTC m=+23.741284411"
Jul 10 14:35:26.350919 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.350891    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 14:35:26.714288 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.714180    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T14:35:26.350913094Z","UUID":"8b0180c4-a329-434c-83c2-dfd515349e0f","Handler":null,"Name":"","Endpoint":""}
Jul 10 14:35:26.715580 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.715562    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 14:35:26.715680 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.715599    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 14:35:26.840095 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:26.840066    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:26.840315 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:26.840188    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:27.024962 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:27.024883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl" event={"ID":"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1","Type":"ContainerStarted","Data":"685dd47dadcc3044418b5263894d14a0e64b63c39c9a695e04b0a0708c43015e"}
Jul 10 14:35:27.026697 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:27.026659    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-62n4t" event={"ID":"5fdbffb2-cc33-4475-b081-9ab070bcb71b","Type":"ContainerStarted","Data":"568120f2ab06caffc578f904e35fdfaa3ffdbe7f6e7c3f1be6611e0fd920cdb3"}
Jul 10 14:35:27.044944 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:27.044894    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-62n4t" podStartSLOduration=3.534597441 podStartE2EDuration="24.044876361s" totalImagesPullingTime="20.51027892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:27.044400233 +0000 UTC m=+24.750721702" watchObservedRunningTime="2026-07-10 14:35:27.044876361 +0000 UTC m=+24.751197829"
Jul 10 14:35:27.840534 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:27.840450    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:27.840692 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:27.840450    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:27.840692 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:27.840565    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:27.840692 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:27.840613    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:28.032277 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:28.032251    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:35:28.032802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:28.032670    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"d4fef2c54a86b57d0e5b8dd69aa4c5e19245e37947158ee207910ae0d45dff56"}
Jul 10 14:35:28.034916 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:28.034885    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl" event={"ID":"8e7f9fd7-3204-4dd0-a791-a7ac41c068c1","Type":"ContainerStarted","Data":"ba6b015594e0c2c38e9776d7e7cadf36026d72edfa4ad0447c563f436b3e0a42"}
Jul 10 14:35:28.058365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:28.058292    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8dpl" podStartSLOduration=2.267265395 podStartE2EDuration="25.058271846s" totalImagesPullingTime="22.791006451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:28.057823198 +0000 UTC m=+25.764144693" watchObservedRunningTime="2026-07-10 14:35:28.058271846 +0000 UTC m=+25.764593319"
Jul 10 14:35:28.840816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:28.840634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:28.841107 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:28.840922    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:29.840372 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:29.840335    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:29.841000 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:29.840390    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:29.841000 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:29.840508    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:29.841000 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:29.840653    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:29.893899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:29.893821    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:29.894533 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:29.894493    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:30.040281 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:30.040247    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:30.040815 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:30.040794    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-25t26"
Jul 10 14:35:30.840269 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:30.840194    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:30.840430 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:30.840326    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:31.840214 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:31.839975    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:31.840350 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:31.839975    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:31.840350 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:31.840254    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:31.840350 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:31.840286    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:32.047835 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.047808    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:35:32.048308 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.048180    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"2426cd2722861a80db96454c648f2dad21012f405e2c22fa6bc8e8390e6db636"}
Jul 10 14:35:32.048460 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.048440    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:32.048562 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.048532    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:32.048659 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.048643    2583 scope.go:117] "RemoveContainer" containerID="203649f7d0b71a816c0acdb6eb579fd60173bf78868444af08ce0a1c7b9f8f29"
Jul 10 14:35:32.050168 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.050145    2583 generic.go:356] "Generic (PLEG): container finished" podID="44846e01-1b2b-453b-ab4a-68fd1b2a28e7" containerID="6d6782786af88ccdc0e989d22590be48cf0d519f892231da78ecf2c39ea0318f" exitCode=0
Jul 10 14:35:32.050264 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.050192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerDied","Data":"6d6782786af88ccdc0e989d22590be48cf0d519f892231da78ecf2c39ea0318f"}
Jul 10 14:35:32.064564 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.064544    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:32.841401 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:32.841366    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:32.841638 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:32.841495    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:33.056175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.056152    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:35:33.056536 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.056473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" event={"ID":"43fa7fd7-d317-44b9-91b8-23fb1f464ad2","Type":"ContainerStarted","Data":"6be5e666a363ec000d766057cf7d0745a608d807413d5d0e77123aa8b3e15b26"}
Jul 10 14:35:33.056671 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.056657    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:33.072795 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.072770    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:35:33.092729 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.092666    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s8565" podStartSLOduration=9.581947857 podStartE2EDuration="30.092652699s" totalImagesPullingTime="20.510704842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:33.091083676 +0000 UTC m=+30.797405139" watchObservedRunningTime="2026-07-10 14:35:33.092652699 +0000 UTC m=+30.798974168"
Jul 10 14:35:33.487589 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.487559    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b58rl"]
Jul 10 14:35:33.487780 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.487677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:33.487780 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:33.487760    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:33.492002 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.491958    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-clcb4"]
Jul 10 14:35:33.492134 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.492079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:33.492205 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:33.492183    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:33.492630 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.492605    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dwdtq"]
Jul 10 14:35:33.492748 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:33.492721    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:33.492849 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:33.492823    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:34.060675 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:34.060589    2583 generic.go:356] "Generic (PLEG): container finished" podID="44846e01-1b2b-453b-ab4a-68fd1b2a28e7" containerID="13a54a3e81ffcbc11125ef489bec2dfa4c22b5fdfffe76cbb5909bae0133149a" exitCode=0
Jul 10 14:35:34.061101 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:34.060670    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerDied","Data":"13a54a3e81ffcbc11125ef489bec2dfa4c22b5fdfffe76cbb5909bae0133149a"}
Jul 10 14:35:34.840398 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:34.840366    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:34.840584 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:34.840479    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:35.065172 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.065022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerStarted","Data":"abc9f2c93f52ec12160cd7566da6794ccfc218d3111ffa4427b407eb3dbddcde"}
Jul 10 14:35:35.500486 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.500459    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:35.500643 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.500508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:35.500643 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.500612    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:35.500721 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.500663    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs podName:a887d595-6f78-4991-9532-c46e7bf126b4 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:07.500649737 +0000 UTC m=+65.206971184 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs") pod "network-metrics-daemon-dwdtq" (UID: "a887d595-6f78-4991-9532-c46e7bf126b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:35:35.500721 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.500612    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:35.500785 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.500741    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret podName:ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf nodeName:}" failed. No retries permitted until 2026-07-10 14:36:07.500727329 +0000 UTC m=+65.207048782 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret") pod "global-pull-secret-syncer-clcb4" (UID: "ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:35:35.600874 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.600806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:35.601010 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.600915    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:35:35.601010 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.600930    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:35:35.601010 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.600940    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8qt2n for pod openshift-network-diagnostics/network-check-target-b58rl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:35.601010 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.600985    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n podName:ba355968-3747-4784-bcff-1d6bd23f1813 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:07.600972981 +0000 UTC m=+65.307294427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8qt2n" (UniqueName: "kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n") pod "network-check-target-b58rl" (UID: "ba355968-3747-4784-bcff-1d6bd23f1813") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:35:35.605882 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.605856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:35.739437 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.739406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:35.840404 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.840374    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:35.840556 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.840374    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:35.840556 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.840483    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dwdtq" podUID="a887d595-6f78-4991-9532-c46e7bf126b4"
Jul 10 14:35:35.840649 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:35.840556    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-clcb4" podUID="ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf"
Jul 10 14:35:35.871047 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:35.870977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:36.056182 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:36.056148    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:36.068903 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:36.068877    2583 generic.go:356] "Generic (PLEG): container finished" podID="44846e01-1b2b-453b-ab4a-68fd1b2a28e7" containerID="abc9f2c93f52ec12160cd7566da6794ccfc218d3111ffa4427b407eb3dbddcde" exitCode=0
Jul 10 14:35:36.069226 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:36.068913    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerDied","Data":"abc9f2c93f52ec12160cd7566da6794ccfc218d3111ffa4427b407eb3dbddcde"}
Jul 10 14:35:36.840717 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:36.840685    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:36.840898 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:36.840815    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b58rl" podUID="ba355968-3747-4784-bcff-1d6bd23f1813"
Jul 10 14:35:37.584991 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.584910    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-210.ec2.internal" event="NodeReady"
Jul 10 14:35:37.585560 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.585071    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 14:35:37.621825 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.621789    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-559554586f-bgs9g"]
Jul 10 14:35:37.623810 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.623793    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.626919 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.626899    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 14:35:37.627103 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.627088    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5vrwq\""
Jul 10 14:35:37.627188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.627164    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 14:35:37.627240 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.627189    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 14:35:37.632816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.632794    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vdw48"]
Jul 10 14:35:37.634189 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.634169    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 14:35:37.634555 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.634537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.637477 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.637461    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 14:35:37.638441 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.638423    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwwtb\""
Jul 10 14:35:37.638902 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.638697    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 14:35:37.639009 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.639000    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-82rgq"]
Jul 10 14:35:37.642257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.642237    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559554586f-bgs9g"]
Jul 10 14:35:37.642361 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.642346    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.645379 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.645360    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 14:35:37.645490 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.645401    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 14:35:37.645490 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.645479    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-crmm7\""
Jul 10 14:35:37.645601 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.645552    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 14:35:37.648006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.647985    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vdw48"]
Jul 10 14:35:37.652233 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.652215    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-82rgq"]
Jul 10 14:35:37.716362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716316    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.716362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd7l2\" (UniqueName: \"kubernetes.io/projected/92643ff5-ab30-43fd-8106-4abb0940b6cd-kube-api-access-kd7l2\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cms5v\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-kube-api-access-cms5v\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92643ff5-ab30-43fd-8106-4abb0940b6cd-config-volume\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-bound-sa-token\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716501    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92643ff5-ab30-43fd-8106-4abb0940b6cd-tmp-dir\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddc0763c-c27d-485d-a69b-c8187a209a09-ca-trust-extracted\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716562    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-certificates\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716597    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-image-registry-private-configuration\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716970 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716674    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-installation-pull-secrets\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716970 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716738    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.716970 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.716764    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-trusted-ca\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.817856 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.817815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddc0763c-c27d-485d-a69b-c8187a209a09-ca-trust-extracted\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.817856 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.817852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-certificates\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818100 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.817873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-image-registry-private-configuration\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818100 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-installation-pull-secrets\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818100 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818100 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-trusted-ca\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:37.818161    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:37.818182    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559554586f-bgs9g: secret "image-registry-tls" not found
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a6052cf-119f-45ad-afe3-ac6e322a8433-cert\") pod \"ingress-canary-82rgq\" (UID: \"3a6052cf-119f-45ad-afe3-ac6e322a8433\") " pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:37.818241    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls podName:ddc0763c-c27d-485d-a69b-c8187a209a09 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:38.318222308 +0000 UTC m=+36.024543754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls") pod "image-registry-559554586f-bgs9g" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09") : secret "image-registry-tls" not found
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddc0763c-c27d-485d-a69b-c8187a209a09-ca-trust-extracted\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.818291 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd7l2\" (UniqueName: \"kubernetes.io/projected/92643ff5-ab30-43fd-8106-4abb0940b6cd-kube-api-access-kd7l2\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:37.818326    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:37.818358    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls podName:92643ff5-ab30-43fd-8106-4abb0940b6cd nodeName:}" failed. No retries permitted until 2026-07-10 14:35:38.318348263 +0000 UTC m=+36.024669737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls") pod "dns-default-vdw48" (UID: "92643ff5-ab30-43fd-8106-4abb0940b6cd") : secret "dns-default-metrics-tls" not found
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp2rr\" (UniqueName: \"kubernetes.io/projected/3a6052cf-119f-45ad-afe3-ac6e322a8433-kube-api-access-rp2rr\") pod \"ingress-canary-82rgq\" (UID: \"3a6052cf-119f-45ad-afe3-ac6e322a8433\") " pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cms5v\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-kube-api-access-cms5v\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92643ff5-ab30-43fd-8106-4abb0940b6cd-config-volume\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-bound-sa-token\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92643ff5-ab30-43fd-8106-4abb0940b6cd-tmp-dir\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.818954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818889    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-certificates\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.818954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.818897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92643ff5-ab30-43fd-8106-4abb0940b6cd-tmp-dir\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.819044 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.819021    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-trusted-ca\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.819668 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.819647    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92643ff5-ab30-43fd-8106-4abb0940b6cd-config-volume\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.822070 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.822042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-image-registry-private-configuration\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.822152 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.822067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-installation-pull-secrets\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.827894 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.827867    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd7l2\" (UniqueName: \"kubernetes.io/projected/92643ff5-ab30-43fd-8106-4abb0940b6cd-kube-api-access-kd7l2\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:37.828144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.828123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-bound-sa-token\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.830967 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.828570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cms5v\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-kube-api-access-cms5v\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:37.840403 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.840350    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:35:37.840542 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.840508    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:35:37.843346 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.843321    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 14:35:37.843346 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.843333    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 14:35:37.843475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.843385    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c8k94\""
Jul 10 14:35:37.919664 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.919635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp2rr\" (UniqueName: \"kubernetes.io/projected/3a6052cf-119f-45ad-afe3-ac6e322a8433-kube-api-access-rp2rr\") pod \"ingress-canary-82rgq\" (UID: \"3a6052cf-119f-45ad-afe3-ac6e322a8433\") " pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.919836 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.919818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a6052cf-119f-45ad-afe3-ac6e322a8433-cert\") pod \"ingress-canary-82rgq\" (UID: \"3a6052cf-119f-45ad-afe3-ac6e322a8433\") " pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.922637 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.922611    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a6052cf-119f-45ad-afe3-ac6e322a8433-cert\") pod \"ingress-canary-82rgq\" (UID: \"3a6052cf-119f-45ad-afe3-ac6e322a8433\") " pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.928843 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.928823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp2rr\" (UniqueName: \"kubernetes.io/projected/3a6052cf-119f-45ad-afe3-ac6e322a8433-kube-api-access-rp2rr\") pod \"ingress-canary-82rgq\" (UID: \"3a6052cf-119f-45ad-afe3-ac6e322a8433\") " pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:37.951842 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:37.951815    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-82rgq"
Jul 10 14:35:38.100451 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.100368    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-82rgq"]
Jul 10 14:35:38.105790 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:35:38.105631    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6052cf_119f_45ad_afe3_ac6e322a8433.slice/crio-669409bae8fe916bb57379a773a37f482e6dcfc4d57e09ef0b559f6ae1bbdc8b WatchSource:0}: Error finding container 669409bae8fe916bb57379a773a37f482e6dcfc4d57e09ef0b559f6ae1bbdc8b: Status 404 returned error can't find the container with id 669409bae8fe916bb57379a773a37f482e6dcfc4d57e09ef0b559f6ae1bbdc8b
Jul 10 14:35:38.322360 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.322326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:38.322532 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.322394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:38.322532 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:38.322483    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 14:35:38.322532 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:38.322500    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 14:35:38.322532 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:38.322510    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559554586f-bgs9g: secret "image-registry-tls" not found
Jul 10 14:35:38.322667 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:38.322568    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls podName:92643ff5-ab30-43fd-8106-4abb0940b6cd nodeName:}" failed. No retries permitted until 2026-07-10 14:35:39.322549888 +0000 UTC m=+37.028871352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls") pod "dns-default-vdw48" (UID: "92643ff5-ab30-43fd-8106-4abb0940b6cd") : secret "dns-default-metrics-tls" not found
Jul 10 14:35:38.322667 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:38.322583    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls podName:ddc0763c-c27d-485d-a69b-c8187a209a09 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:39.322577191 +0000 UTC m=+37.028898638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls") pod "image-registry-559554586f-bgs9g" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09") : secret "image-registry-tls" not found
Jul 10 14:35:38.440758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.440725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:38.541072 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.541012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:38.840334 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.840259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:35:38.843842 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.843697    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w457z\""
Jul 10 14:35:38.845135 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.845113    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 14:35:38.845250 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:38.845144    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 14:35:39.078695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:39.078656    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-82rgq" event={"ID":"3a6052cf-119f-45ad-afe3-ac6e322a8433","Type":"ContainerStarted","Data":"669409bae8fe916bb57379a773a37f482e6dcfc4d57e09ef0b559f6ae1bbdc8b"}
Jul 10 14:35:39.331448 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:39.331409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:39.331649 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:39.331540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:39.331649 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:39.331557    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 14:35:39.331649 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:39.331636    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls podName:92643ff5-ab30-43fd-8106-4abb0940b6cd nodeName:}" failed. No retries permitted until 2026-07-10 14:35:41.331616079 +0000 UTC m=+39.037937540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls") pod "dns-default-vdw48" (UID: "92643ff5-ab30-43fd-8106-4abb0940b6cd") : secret "dns-default-metrics-tls" not found
Jul 10 14:35:39.331808 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:39.331654    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 14:35:39.331808 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:39.331666    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559554586f-bgs9g: secret "image-registry-tls" not found
Jul 10 14:35:39.331808 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:39.331698    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls podName:ddc0763c-c27d-485d-a69b-c8187a209a09 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:41.331688332 +0000 UTC m=+39.038009796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls") pod "image-registry-559554586f-bgs9g" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09") : secret "image-registry-tls" not found
Jul 10 14:35:40.321460 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:40.321374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:40.386133 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:40.386096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:41.348386 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:41.348350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:41.348823 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:41.348412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:41.348823 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:41.348529    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 14:35:41.348823 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:41.348553    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559554586f-bgs9g: secret "image-registry-tls" not found
Jul 10 14:35:41.348823 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:41.348569    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 14:35:41.348823 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:41.348621    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls podName:ddc0763c-c27d-485d-a69b-c8187a209a09 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:45.348599101 +0000 UTC m=+43.054920550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls") pod "image-registry-559554586f-bgs9g" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09") : secret "image-registry-tls" not found
Jul 10 14:35:41.348823 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:41.348640    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls podName:92643ff5-ab30-43fd-8106-4abb0940b6cd nodeName:}" failed. No retries permitted until 2026-07-10 14:35:45.348631047 +0000 UTC m=+43.054952499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls") pod "dns-default-vdw48" (UID: "92643ff5-ab30-43fd-8106-4abb0940b6cd") : secret "dns-default-metrics-tls" not found
Jul 10 14:35:42.542723 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:42.542691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:42.647456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:42.647378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:43.089811 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:43.089782    2583 generic.go:356] "Generic (PLEG): container finished" podID="44846e01-1b2b-453b-ab4a-68fd1b2a28e7" containerID="aeae27de903177818b892c50aeeab49993b6c7aa8e105181eb008fab34fd5588" exitCode=0
Jul 10 14:35:43.089951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:43.089856    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerDied","Data":"aeae27de903177818b892c50aeeab49993b6c7aa8e105181eb008fab34fd5588"}
Jul 10 14:35:43.091286 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:43.091264    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-82rgq" event={"ID":"3a6052cf-119f-45ad-afe3-ac6e322a8433","Type":"ContainerStarted","Data":"f8a047fddd9d33807602f859686c3db1db634451352c6bb7dbae6b95281fe88e"}
Jul 10 14:35:43.135259 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:43.135210    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-82rgq" podStartSLOduration=2.191841841 podStartE2EDuration="6.135189778s" totalImagesPullingTime="3.943347937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:43.134397759 +0000 UTC m=+40.840719228" watchObservedRunningTime="2026-07-10 14:35:43.135189778 +0000 UTC m=+40.841511248"
Jul 10 14:35:44.095974 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:44.095944    2583 generic.go:356] "Generic (PLEG): container finished" podID="44846e01-1b2b-453b-ab4a-68fd1b2a28e7" containerID="99f95bc7a089e46858d81b14fa9a7876ff174e4cf78fd9d2e6754d888a8eaece" exitCode=0
Jul 10 14:35:44.096355 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:44.096034    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerDied","Data":"99f95bc7a089e46858d81b14fa9a7876ff174e4cf78fd9d2e6754d888a8eaece"}
Jul 10 14:35:44.583334 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:44.583299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:44.647003 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:44.646967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:45.068280 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.068249    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rfr2g"]
Jul 10 14:35:45.088113 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.088074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.088753 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.088726    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rfr2g"]
Jul 10 14:35:45.090549 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.090529    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmfjj\""
Jul 10 14:35:45.090656 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.090636    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 14:35:45.091596 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.091577    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 14:35:45.091694 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.091606    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 14:35:45.091694 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.091582    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 14:35:45.101425 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.101398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" event={"ID":"44846e01-1b2b-453b-ab4a-68fd1b2a28e7","Type":"ContainerStarted","Data":"60814d3fc444530d0419274d84300fae4a05af6b003c750546bf59986120f68d"}
Jul 10 14:35:45.129981 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.129933    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dqlw9" podStartSLOduration=8.221239604 podStartE2EDuration="42.129917235s" totalImagesPullingTime="33.908677631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:45.128915526 +0000 UTC m=+42.835236995" watchObservedRunningTime="2026-07-10 14:35:45.129917235 +0000 UTC m=+42.836238704"
Jul 10 14:35:45.252149 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.252121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:45.280210 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.280177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-crio-socket\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.280343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.280306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-data-volume\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.280403 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.280350    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.280472 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.280450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.280551 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.280505    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgqsb\" (UniqueName: \"kubernetes.io/projected/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-api-access-kgqsb\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.333614 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.333540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:45.381853 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.381822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-crio-socket\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.381950 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.381865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:45.382002 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.381969    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 14:35:45.382050 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382016    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls podName:92643ff5-ab30-43fd-8106-4abb0940b6cd nodeName:}" failed. No retries permitted until 2026-07-10 14:35:53.382002089 +0000 UTC m=+51.088323537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls") pod "dns-default-vdw48" (UID: "92643ff5-ab30-43fd-8106-4abb0940b6cd") : secret "dns-default-metrics-tls" not found
Jul 10 14:35:45.382050 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382011    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-data-volume\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.382171 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.382171 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382134    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 14:35:45.382171 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382066    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-crio-socket\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382195    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:45.882177788 +0000 UTC m=+43.588499258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : configmap "kube-rbac-proxy" not found
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382223    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382237    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559554586f-bgs9g: secret "image-registry-tls" not found
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382271    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgqsb\" (UniqueName: \"kubernetes.io/projected/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-api-access-kgqsb\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.382302 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382281    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls podName:ddc0763c-c27d-485d-a69b-c8187a209a09 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:53.382265943 +0000 UTC m=+51.088587402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls") pod "image-registry-559554586f-bgs9g" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09") : secret "image-registry-tls" not found
Jul 10 14:35:45.382554 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382338    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:45.382554 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.382335    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-data-volume\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.382554 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.382381    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:45.882369959 +0000 UTC m=+43.588691410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:45.393056 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.393026    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgqsb\" (UniqueName: \"kubernetes.io/projected/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-api-access-kgqsb\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.676046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.676015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:45.746802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.746767    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"]
Jul 10 14:35:45.769182 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.769150    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:45.769301 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.769211    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"]
Jul 10 14:35:45.771912 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.771842    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 14:35:45.771912 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.771865    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 14:35:45.772115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.771998    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sdxsj\""
Jul 10 14:35:45.788225 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.788202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:45.886354 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.886309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.886507 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.886383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:45.886507 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.886428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:45.886507 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.886456    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:45.886507 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.886464    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:45.886730 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.886541    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:46.886508565 +0000 UTC m=+44.592830013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:45.886730 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.886587    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 14:35:45.886730 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.886648    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:46.88663064 +0000 UTC m=+44.592952097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : configmap "kube-rbac-proxy" not found
Jul 10 14:35:45.952370 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.952293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:45.986968 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.986942    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:45.987072 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.987008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:45.987135 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.987082    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 14:35:45.987173 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:45.987137    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert podName:08c26ab0-91eb-441c-9a0b-858c6fb2fa43 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:46.487122452 +0000 UTC m=+44.193443899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bm7wb" (UID: "08c26ab0-91eb-441c-9a0b-858c6fb2fa43") : secret "networking-console-plugin-cert" not found
Jul 10 14:35:45.987634 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:45.987618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:46.031013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.030978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:46.453641 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.453616    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-prtg5_5944324d-e133-48a0-920c-26ae31508a68/dns-node-resolver/0.log"
Jul 10 14:35:46.492939 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.492916    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:46.493082 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:46.493065    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 14:35:46.493150 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:46.493138    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert podName:08c26ab0-91eb-441c-9a0b-858c6fb2fa43 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:47.493119892 +0000 UTC m=+45.199441339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bm7wb" (UID: "08c26ab0-91eb-441c-9a0b-858c6fb2fa43") : secret "networking-console-plugin-cert" not found
Jul 10 14:35:46.692207 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.692174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:46.789657 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.789584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:46.896127 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.896092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:46.896289 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.896195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:46.896289 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:46.896229    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:46.896289 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:46.896284    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 14:35:46.896389 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:46.896289    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:48.896274098 +0000 UTC m=+46.602595545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:46.896389 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:46.896350    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:48.896334676 +0000 UTC m=+46.602656124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : configmap "kube-rbac-proxy" not found
Jul 10 14:35:46.928574 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:46.928548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:47.020413 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:47.020387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:47.453345 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:47.453317    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-474nb_3055e9b1-3af7-4d56-83ca-574b55464161/node-ca/0.log"
Jul 10 14:35:47.499846 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:47.499816    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:47.500165 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:47.499958    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 14:35:47.500165 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:47.500018    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert podName:08c26ab0-91eb-441c-9a0b-858c6fb2fa43 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:49.500004189 +0000 UTC m=+47.206325637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bm7wb" (UID: "08c26ab0-91eb-441c-9a0b-858c6fb2fa43") : secret "networking-console-plugin-cert" not found
Jul 10 14:35:48.052552 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:48.052500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:48.252334 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:48.252310    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-82rgq_3a6052cf-119f-45ad-afe3-ac6e322a8433/serve-healthcheck-canary/0.log"
Jul 10 14:35:48.908378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:48.908342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:48.908935 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:48.908398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:48.908935 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:48.908462    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 14:35:48.908935 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:48.908539    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:52.908508369 +0000 UTC m=+50.614829816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : configmap "kube-rbac-proxy" not found
Jul 10 14:35:48.908935 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:48.908558    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:48.908935 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:48.908608    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:52.908593916 +0000 UTC m=+50.614915364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:49.512098 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:49.512057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:49.512283 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:49.512192    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 14:35:49.512283 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:49.512258    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert podName:08c26ab0-91eb-441c-9a0b-858c6fb2fa43 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:53.512243361 +0000 UTC m=+51.218564808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bm7wb" (UID: "08c26ab0-91eb-441c-9a0b-858c6fb2fa43") : secret "networking-console-plugin-cert" not found
Jul 10 14:35:51.020536 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:51.020493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:51.135013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:51.134983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:51.365403 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:51.365317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:52.302311 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:52.302277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:52.510696 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:52.510662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:52.695465 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:52.695437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:52.789176 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:52.789143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:52.940483 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:52.940449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:52.940666 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:52.940501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:35:52.940666 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:52.940582    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 14:35:52.940666 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:52.940613    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:52.940666 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:52.940641    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:00.94062636 +0000 UTC m=+58.646947808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : configmap "kube-rbac-proxy" not found
Jul 10 14:35:52.940666 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:52.940654    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:00.940647859 +0000 UTC m=+58.646969305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : secret "insights-runtime-extractor-tls" not found
Jul 10 14:35:53.023856 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:53.023781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:53.444533 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:53.444486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:35:53.444894 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:53.444597    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:35:53.444894 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.444640    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 14:35:53.444894 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.444697    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls podName:92643ff5-ab30-43fd-8106-4abb0940b6cd nodeName:}" failed. No retries permitted until 2026-07-10 14:36:09.444682722 +0000 UTC m=+67.151004170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls") pod "dns-default-vdw48" (UID: "92643ff5-ab30-43fd-8106-4abb0940b6cd") : secret "dns-default-metrics-tls" not found
Jul 10 14:35:53.444894 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.444713    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 14:35:53.444894 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.444725    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559554586f-bgs9g: secret "image-registry-tls" not found
Jul 10 14:35:53.444894 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.444767    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls podName:ddc0763c-c27d-485d-a69b-c8187a209a09 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:09.444751544 +0000 UTC m=+67.151072991 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls") pod "image-registry-559554586f-bgs9g" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09") : secret "image-registry-tls" not found
Jul 10 14:35:53.544941 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:53.544909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:35:53.545104 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.545021    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 14:35:53.545104 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:35:53.545083    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert podName:08c26ab0-91eb-441c-9a0b-858c6fb2fa43 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:01.545066003 +0000 UTC m=+59.251387451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bm7wb" (UID: "08c26ab0-91eb-441c-9a0b-858c6fb2fa43") : secret "networking-console-plugin-cert" not found
Jul 10 14:35:53.725763 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:53.725687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.121423 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:54.121338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.261039 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:54.261005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.405651 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:54.405565    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.940936 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:54.940901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.360354 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:55.360271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.454454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:55.454425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.721611 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:55.721582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.924398 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:55.924365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.994993 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:55.994921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:56.330301 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:56.330224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:56.686873 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:56.686845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:57.127460 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:57.127371    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:57.336057 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:57.336027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:57.715750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:57.715721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:57.926957 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:57.926926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:58.130618 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:58.130551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:58.319282 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:58.319255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:58.916257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:58.916227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:59.123556 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:59.123508    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:59.520707 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:59.520681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:59.718888 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:59.718854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:59.816815 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:59.816744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:59.916639 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:35:59.916608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:00.315860 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:00.315830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:01.000037 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:00.999998    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:36:01.000428 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.000056    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:36:01.000428 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:01.000180    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 14:36:01.000428 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:01.000241    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls podName:657b971b-c38c-45c7-a2e8-f47c4c2f0f68 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:17.000227027 +0000 UTC m=+74.706548486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rfr2g" (UID: "657b971b-c38c-45c7-a2e8-f47c4c2f0f68") : secret "insights-runtime-extractor-tls" not found
Jul 10 14:36:01.000566 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.000506    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:36:01.111692 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.111658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:01.523741 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.523711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:01.605644 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.605602    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:36:01.608034 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.608011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c26ab0-91eb-441c-9a0b-858c6fb2fa43-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bm7wb\" (UID: \"08c26ab0-91eb-441c-9a0b-858c6fb2fa43\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:36:01.679025 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.679000    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"
Jul 10 14:36:01.720177 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.720149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:01.801749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.801669    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bm7wb"]
Jul 10 14:36:01.805780 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:01.805753    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c26ab0_91eb_441c_9a0b_858c6fb2fa43.slice/crio-41dfbdf874f7ec73acc85b435212ac6c216aa203e995c706d99c7cc0b7fa05cb WatchSource:0}: Error finding container 41dfbdf874f7ec73acc85b435212ac6c216aa203e995c706d99c7cc0b7fa05cb: Status 404 returned error can't find the container with id 41dfbdf874f7ec73acc85b435212ac6c216aa203e995c706d99c7cc0b7fa05cb
Jul 10 14:36:01.943884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:01.943853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:02.139949 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:02.139862    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb" event={"ID":"08c26ab0-91eb-441c-9a0b-858c6fb2fa43","Type":"ContainerStarted","Data":"41dfbdf874f7ec73acc85b435212ac6c216aa203e995c706d99c7cc0b7fa05cb"}
Jul 10 14:36:02.382993 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:02.382957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:02.721905 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:02.721860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:02.918893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:02.918862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:04.146030 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:04.145997    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb" event={"ID":"08c26ab0-91eb-441c-9a0b-858c6fb2fa43","Type":"ContainerStarted","Data":"ff64aa1b06ab5de37ad880b6e4cf6e4d2e9f625a95ef1387bd03bd2b8d57abb7"}
Jul 10 14:36:04.174733 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:04.174691    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-bm7wb" podStartSLOduration=17.355518483 podStartE2EDuration="19.174677995s" totalImagesPullingTime="1.819159512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:04.174441319 +0000 UTC m=+61.880762788" watchObservedRunningTime="2026-07-10 14:36:04.174677995 +0000 UTC m=+61.880999463"
Jul 10 14:36:04.789369 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:04.789339    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:04.889481 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:04.889446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:04.990447 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:04.990417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:05.075866 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:05.075800    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8565"
Jul 10 14:36:05.090754 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:05.090728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:05.155045 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:05.155009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:05.275765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:05.275732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:05.334594 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:05.334478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:06.106183 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:06.106153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:06.208159 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:06.208130    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:06.679141 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:06.679101    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:06.880281 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:06.880248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:06.977545 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:06.977451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:07.550290 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.550257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:36:07.550670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.550309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:36:07.552954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.552937    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 14:36:07.552999 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.552954    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 14:36:07.563370 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.563338    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a887d595-6f78-4991-9532-c46e7bf126b4-metrics-certs\") pod \"network-metrics-daemon-dwdtq\" (UID: \"a887d595-6f78-4991-9532-c46e7bf126b4\") " pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:36:07.563491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.563418    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf-original-pull-secret\") pod \"global-pull-secret-syncer-clcb4\" (UID: \"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf\") " pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:36:07.651287 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.651255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:36:07.653822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.653807    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 14:36:07.664293 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.664275    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 14:36:07.674885 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.674865    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qt2n\" (UniqueName: \"kubernetes.io/projected/ba355968-3747-4784-bcff-1d6bd23f1813-kube-api-access-8qt2n\") pod \"network-check-target-b58rl\" (UID: \"ba355968-3747-4784-bcff-1d6bd23f1813\") " pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:36:07.849784 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.849713    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-clcb4"
Jul 10 14:36:07.856674 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.856655    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c8k94\""
Jul 10 14:36:07.865022 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.865001    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dwdtq"
Jul 10 14:36:07.955598 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.955542    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w457z\""
Jul 10 14:36:07.964006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.963955    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:36:07.979087 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.979058    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-clcb4"]
Jul 10 14:36:07.982094 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:07.982044    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9c54c3_88af_4d02_8935_2a8c8eeb5dbf.slice/crio-d731ca00652ad1dddaa21ee5212b86108bf18585ad482eca0a05e7bbc05da648 WatchSource:0}: Error finding container d731ca00652ad1dddaa21ee5212b86108bf18585ad482eca0a05e7bbc05da648: Status 404 returned error can't find the container with id d731ca00652ad1dddaa21ee5212b86108bf18585ad482eca0a05e7bbc05da648
Jul 10 14:36:07.993995 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:07.993937    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dwdtq"]
Jul 10 14:36:07.997454 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:07.997429    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda887d595_6f78_4991_9532_c46e7bf126b4.slice/crio-6db7f2cff2e34c22d8f1ea7635dcd89552133efc507422a9b39c40d4e5caf3cb WatchSource:0}: Error finding container 6db7f2cff2e34c22d8f1ea7635dcd89552133efc507422a9b39c40d4e5caf3cb: Status 404 returned error can't find the container with id 6db7f2cff2e34c22d8f1ea7635dcd89552133efc507422a9b39c40d4e5caf3cb
Jul 10 14:36:08.087134 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:08.087105    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b58rl"]
Jul 10 14:36:08.090537 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:08.090493    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba355968_3747_4784_bcff_1d6bd23f1813.slice/crio-5019a318d20d2eee7392c51f94dfcd339d1882dbe9d98e585b133a70358cd484 WatchSource:0}: Error finding container 5019a318d20d2eee7392c51f94dfcd339d1882dbe9d98e585b133a70358cd484: Status 404 returned error can't find the container with id 5019a318d20d2eee7392c51f94dfcd339d1882dbe9d98e585b133a70358cd484
Jul 10 14:36:08.157059 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:08.156979    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b58rl" event={"ID":"ba355968-3747-4784-bcff-1d6bd23f1813","Type":"ContainerStarted","Data":"5019a318d20d2eee7392c51f94dfcd339d1882dbe9d98e585b133a70358cd484"}
Jul 10 14:36:08.158150 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:08.158123    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dwdtq" event={"ID":"a887d595-6f78-4991-9532-c46e7bf126b4","Type":"ContainerStarted","Data":"6db7f2cff2e34c22d8f1ea7635dcd89552133efc507422a9b39c40d4e5caf3cb"}
Jul 10 14:36:08.159140 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:08.159120    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-clcb4" event={"ID":"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf","Type":"ContainerStarted","Data":"d731ca00652ad1dddaa21ee5212b86108bf18585ad482eca0a05e7bbc05da648"}
Jul 10 14:36:09.464505 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.464473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:36:09.464901 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.464570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:09.467958 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.467861    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92643ff5-ab30-43fd-8106-4abb0940b6cd-metrics-tls\") pod \"dns-default-vdw48\" (UID: \"92643ff5-ab30-43fd-8106-4abb0940b6cd\") " pod="openshift-dns/dns-default-vdw48"
Jul 10 14:36:09.468495 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.468471    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"image-registry-559554586f-bgs9g\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") " pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:09.736031 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.735989    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5vrwq\""
Jul 10 14:36:09.743588 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.743536    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:09.746454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.746430    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwwtb\""
Jul 10 14:36:09.753968 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.753944    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdw48"
Jul 10 14:36:09.911106 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.910999    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559554586f-bgs9g"]
Jul 10 14:36:09.925123 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:09.925099    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vdw48"]
Jul 10 14:36:09.931615 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:09.931581    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92643ff5_ab30_43fd_8106_4abb0940b6cd.slice/crio-ada9c7aa0e1977e066ebd97259267cb784d78393051fa0c82e0b26e67ba5d2b5 WatchSource:0}: Error finding container ada9c7aa0e1977e066ebd97259267cb784d78393051fa0c82e0b26e67ba5d2b5: Status 404 returned error can't find the container with id ada9c7aa0e1977e066ebd97259267cb784d78393051fa0c82e0b26e67ba5d2b5
Jul 10 14:36:10.174266 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.174151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dwdtq" event={"ID":"a887d595-6f78-4991-9532-c46e7bf126b4","Type":"ContainerStarted","Data":"614026e96ae7249bedfb1d8288830926117991a2e77cad3c0f67f0064bfdd05d"}
Jul 10 14:36:10.174266 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.174221    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dwdtq" event={"ID":"a887d595-6f78-4991-9532-c46e7bf126b4","Type":"ContainerStarted","Data":"c3e012b2a265ae3a5d5c8815970348c8cf0713b3b2f7ce81f064e057f4e2d29f"}
Jul 10 14:36:10.176325 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.176290    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559554586f-bgs9g" event={"ID":"ddc0763c-c27d-485d-a69b-c8187a209a09","Type":"ContainerStarted","Data":"3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e"}
Jul 10 14:36:10.176452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.176334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559554586f-bgs9g" event={"ID":"ddc0763c-c27d-485d-a69b-c8187a209a09","Type":"ContainerStarted","Data":"fc90ba8dbbc48e8e143921a93d8238711bb1387cceee6a3400f83904551f826b"}
Jul 10 14:36:10.176452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.176366    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:10.178164 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.178129    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdw48" event={"ID":"92643ff5-ab30-43fd-8106-4abb0940b6cd","Type":"ContainerStarted","Data":"ada9c7aa0e1977e066ebd97259267cb784d78393051fa0c82e0b26e67ba5d2b5"}
Jul 10 14:36:10.193654 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.193606    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dwdtq" podStartSLOduration=65.911560646 podStartE2EDuration="1m7.193591359s" totalImagesPullingTime="1.282030713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:10.193197557 +0000 UTC m=+67.899519026" watchObservedRunningTime="2026-07-10 14:36:10.193591359 +0000 UTC m=+67.899912829"
Jul 10 14:36:10.223166 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.222977    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-559554586f-bgs9g" podStartSLOduration=50.222954894 podStartE2EDuration="50.222954894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:10.220951217 +0000 UTC m=+67.927272688" watchObservedRunningTime="2026-07-10 14:36:10.222954894 +0000 UTC m=+67.929276366"
Jul 10 14:36:10.334036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.333965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:10.410939 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:10.410882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:11.723286 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.723255    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559554586f-bgs9g"]
Jul 10 14:36:11.741068 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.741039    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"]
Jul 10 14:36:11.744007 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.743981    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.747016 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.746861    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-stdx7\""
Jul 10 14:36:11.747016 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.747005    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 14:36:11.749543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.747902    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 10 14:36:11.751538 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.749992    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 14:36:11.751538 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.750103    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 14:36:11.764434 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.764408    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"]
Jul 10 14:36:11.767219 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.767186    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"]
Jul 10 14:36:11.767332 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.767294    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.770028 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.770004    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 10 14:36:11.780708 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.780684    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"]
Jul 10 14:36:11.784590 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.784564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a2eea60-9055-4536-a8d3-a3f90e4915d1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c85879b4-2g2mt\" (UID: \"7a2eea60-9055-4536-a8d3-a3f90e4915d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.784685 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.784622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hswzq\" (UniqueName: \"kubernetes.io/projected/7a2eea60-9055-4536-a8d3-a3f90e4915d1-kube-api-access-hswzq\") pod \"managed-serviceaccount-addon-agent-7c85879b4-2g2mt\" (UID: \"7a2eea60-9055-4536-a8d3-a3f90e4915d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.787589 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.787567    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:11.840184 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.840151    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-wldc5"]
Jul 10 14:36:11.843205 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.843185    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:11.845883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.845863    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pb6kq\""
Jul 10 14:36:11.846235 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.846212    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 14:36:11.846331 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.846247    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 14:36:11.857313 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.857253    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-wldc5"]
Jul 10 14:36:11.885020 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.884995    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f887ff3-e876-4c34-a423-949ac8e56f9c-tmp\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.885150 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.885034    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a2eea60-9055-4536-a8d3-a3f90e4915d1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c85879b4-2g2mt\" (UID: \"7a2eea60-9055-4536-a8d3-a3f90e4915d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.885150 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.885101    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbxfw\" (UniqueName: \"kubernetes.io/projected/801e38db-10ba-44b5-90c4-bcce65f80b3f-kube-api-access-pbxfw\") pod \"downloads-68c9cbb497-wldc5\" (UID: \"801e38db-10ba-44b5-90c4-bcce65f80b3f\") " pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:11.885150 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.885145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hswzq\" (UniqueName: \"kubernetes.io/projected/7a2eea60-9055-4536-a8d3-a3f90e4915d1-kube-api-access-hswzq\") pod \"managed-serviceaccount-addon-agent-7c85879b4-2g2mt\" (UID: \"7a2eea60-9055-4536-a8d3-a3f90e4915d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.885288 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.885168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1f887ff3-e876-4c34-a423-949ac8e56f9c-klusterlet-config\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.885288 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.885202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlfj7\" (UniqueName: \"kubernetes.io/projected/1f887ff3-e876-4c34-a423-949ac8e56f9c-kube-api-access-vlfj7\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.888036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.888015    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a2eea60-9055-4536-a8d3-a3f90e4915d1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c85879b4-2g2mt\" (UID: \"7a2eea60-9055-4536-a8d3-a3f90e4915d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.898416 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.898386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:11.901742 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.901719    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hswzq\" (UniqueName: \"kubernetes.io/projected/7a2eea60-9055-4536-a8d3-a3f90e4915d1-kube-api-access-hswzq\") pod \"managed-serviceaccount-addon-agent-7c85879b4-2g2mt\" (UID: \"7a2eea60-9055-4536-a8d3-a3f90e4915d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:11.985880 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.985794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbxfw\" (UniqueName: \"kubernetes.io/projected/801e38db-10ba-44b5-90c4-bcce65f80b3f-kube-api-access-pbxfw\") pod \"downloads-68c9cbb497-wldc5\" (UID: \"801e38db-10ba-44b5-90c4-bcce65f80b3f\") " pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:11.985880 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.985868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1f887ff3-e876-4c34-a423-949ac8e56f9c-klusterlet-config\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.986103 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.985906    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlfj7\" (UniqueName: \"kubernetes.io/projected/1f887ff3-e876-4c34-a423-949ac8e56f9c-kube-api-access-vlfj7\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.986103 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.985936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f887ff3-e876-4c34-a423-949ac8e56f9c-tmp\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.986393 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.986371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f887ff3-e876-4c34-a423-949ac8e56f9c-tmp\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.988774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.988747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1f887ff3-e876-4c34-a423-949ac8e56f9c-klusterlet-config\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:11.996391 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:11.996362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.002113 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.002094    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbxfw\" (UniqueName: \"kubernetes.io/projected/801e38db-10ba-44b5-90c4-bcce65f80b3f-kube-api-access-pbxfw\") pod \"downloads-68c9cbb497-wldc5\" (UID: \"801e38db-10ba-44b5-90c4-bcce65f80b3f\") " pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:12.002248 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.002230    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlfj7\" (UniqueName: \"kubernetes.io/projected/1f887ff3-e876-4c34-a423-949ac8e56f9c-kube-api-access-vlfj7\") pod \"klusterlet-addon-workmgr-c665b5c75-lqt9g\" (UID: \"1f887ff3-e876-4c34-a423-949ac8e56f9c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:12.068267 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.068239    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"
Jul 10 14:36:12.077301 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.077277    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:12.093452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.093401    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.152566 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.152530    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:12.290409 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.290329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.416735 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.416704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.594646 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.594571    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.657262 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.657232    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"]
Jul 10 14:36:12.660411 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.660390    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:12.665051 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.665029    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 10 14:36:12.667935 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.667869    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wsbvn\""
Jul 10 14:36:12.679188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.679167    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"]
Jul 10 14:36:12.692623 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.692595    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fcd96d02-8d9c-436e-aa44-a7a06b4a049d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nv645\" (UID: \"fcd96d02-8d9c-436e-aa44-a7a06b4a049d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:12.723455 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.723430    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.793995 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.793959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fcd96d02-8d9c-436e-aa44-a7a06b4a049d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nv645\" (UID: \"fcd96d02-8d9c-436e-aa44-a7a06b4a049d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:12.796677 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.796653    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fcd96d02-8d9c-436e-aa44-a7a06b4a049d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nv645\" (UID: \"fcd96d02-8d9c-436e-aa44-a7a06b4a049d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:12.807179 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.807142    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:12.971208 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:12.971165    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:13.544433 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:13.543888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:13.580385 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:13.579921    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt"]
Jul 10 14:36:13.583193 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:13.583167    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a2eea60_9055_4536_a8d3_a3f90e4915d1.slice/crio-18cfcbd811043e8325dcaec863c96b4c4b80ec71937ff208545e4d69d0039781 WatchSource:0}: Error finding container 18cfcbd811043e8325dcaec863c96b4c4b80ec71937ff208545e4d69d0039781: Status 404 returned error can't find the container with id 18cfcbd811043e8325dcaec863c96b4c4b80ec71937ff208545e4d69d0039781
Jul 10 14:36:13.634781 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:13.634539    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-wldc5"]
Jul 10 14:36:13.635828 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:13.635669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:13.640655 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:13.639791    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801e38db_10ba_44b5_90c4_bcce65f80b3f.slice/crio-5e81d329ef0a737fbfc4c15aef2d10c04771160caea9411055efd7a11465b7d3 WatchSource:0}: Error finding container 5e81d329ef0a737fbfc4c15aef2d10c04771160caea9411055efd7a11465b7d3: Status 404 returned error can't find the container with id 5e81d329ef0a737fbfc4c15aef2d10c04771160caea9411055efd7a11465b7d3
Jul 10 14:36:13.800630 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:13.800602    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"]
Jul 10 14:36:13.804367 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:13.804324    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd96d02_8d9c_436e_aa44_a7a06b4a049d.slice/crio-9ce4d959db9f582d4816d4c59ffc3bcabaa35c9da5d110505795592e22b9cdef WatchSource:0}: Error finding container 9ce4d959db9f582d4816d4c59ffc3bcabaa35c9da5d110505795592e22b9cdef: Status 404 returned error can't find the container with id 9ce4d959db9f582d4816d4c59ffc3bcabaa35c9da5d110505795592e22b9cdef
Jul 10 14:36:13.804937 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:13.804903    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"]
Jul 10 14:36:13.807470 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:13.807439    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f887ff3_e876_4c34_a423_949ac8e56f9c.slice/crio-7a051d7f0fee73e4536ba9aefddab0da2e1f86d83a2f079181c022835ebe6b6f WatchSource:0}: Error finding container 7a051d7f0fee73e4536ba9aefddab0da2e1f86d83a2f079181c022835ebe6b6f: Status 404 returned error can't find the container with id 7a051d7f0fee73e4536ba9aefddab0da2e1f86d83a2f079181c022835ebe6b6f
Jul 10 14:36:14.197047 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.197003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdw48" event={"ID":"92643ff5-ab30-43fd-8106-4abb0940b6cd","Type":"ContainerStarted","Data":"e1db310dff20da788842aed6662939e0961868808647573238408dec3c56c2e2"}
Jul 10 14:36:14.197246 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.197062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdw48" event={"ID":"92643ff5-ab30-43fd-8106-4abb0940b6cd","Type":"ContainerStarted","Data":"ba1cc2bd823f9dfd4c5f56cc73101a2ebcd87e429eeb82046d4d3fcdb004347b"}
Jul 10 14:36:14.197246 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.197122    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vdw48"
Jul 10 14:36:14.198620 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.198588    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g" event={"ID":"1f887ff3-e876-4c34-a423-949ac8e56f9c","Type":"ContainerStarted","Data":"7a051d7f0fee73e4536ba9aefddab0da2e1f86d83a2f079181c022835ebe6b6f"}
Jul 10 14:36:14.200035 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.200005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt" event={"ID":"7a2eea60-9055-4536-a8d3-a3f90e4915d1","Type":"ContainerStarted","Data":"18cfcbd811043e8325dcaec863c96b4c4b80ec71937ff208545e4d69d0039781"}
Jul 10 14:36:14.201951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.201892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-clcb4" event={"ID":"ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf","Type":"ContainerStarted","Data":"d11c0ad6577192b770b1ef90c60c7b5a9b96103c99330eccd073fbba1b8bdd9f"}
Jul 10 14:36:14.203660 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.203602    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-wldc5" event={"ID":"801e38db-10ba-44b5-90c4-bcce65f80b3f","Type":"ContainerStarted","Data":"5e81d329ef0a737fbfc4c15aef2d10c04771160caea9411055efd7a11465b7d3"}
Jul 10 14:36:14.205327 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.205286    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b58rl" event={"ID":"ba355968-3747-4784-bcff-1d6bd23f1813","Type":"ContainerStarted","Data":"8bad4d0089329d93c6b5e6b0362e4d1d49e6dd840295bf1c1676cc03523a061a"}
Jul 10 14:36:14.205444 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.205431    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:36:14.207173 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.207148    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645" event={"ID":"fcd96d02-8d9c-436e-aa44-a7a06b4a049d","Type":"ContainerStarted","Data":"9ce4d959db9f582d4816d4c59ffc3bcabaa35c9da5d110505795592e22b9cdef"}
Jul 10 14:36:14.252923 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.252845    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vdw48" podStartSLOduration=33.785761823 podStartE2EDuration="37.252821346s" totalImagesPullingTime="3.467059523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:14.228463541 +0000 UTC m=+71.934785012" watchObservedRunningTime="2026-07-10 14:36:14.252821346 +0000 UTC m=+71.959142817"
Jul 10 14:36:14.282458 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.282386    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b58rl" podStartSLOduration=65.977262289 podStartE2EDuration="1m11.282364101s" totalImagesPullingTime="5.305101812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:14.254639173 +0000 UTC m=+71.960960644" watchObservedRunningTime="2026-07-10 14:36:14.282364101 +0000 UTC m=+71.988685568"
Jul 10 14:36:14.283627 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:14.283585    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-clcb4" podStartSLOduration=65.866213696 podStartE2EDuration="1m11.283567866s" totalImagesPullingTime="5.41735417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:14.282711958 +0000 UTC m=+71.989033427" watchObservedRunningTime="2026-07-10 14:36:14.283567866 +0000 UTC m=+71.989889336"
Jul 10 14:36:15.113417 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:15.113372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:15.193177 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:15.193142    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:15.806469 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:15.806139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:15.878100 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:15.878017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:17.031883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:17.031837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:36:17.034489 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:17.034464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/657b971b-c38c-45c7-a2e8-f47c4c2f0f68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rfr2g\" (UID: \"657b971b-c38c-45c7-a2e8-f47c4c2f0f68\") " pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:36:17.200956 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:17.200915    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmfjj\""
Jul 10 14:36:17.207756 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:17.207725    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rfr2g"
Jul 10 14:36:18.497627 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:18.497573    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rfr2g"]
Jul 10 14:36:18.502682 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:18.502469    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657b971b_c38c_45c7_a2e8_f47c4c2f0f68.slice/crio-3827329b4707c6d72fe10730b0783ae9f8aba3771ab841d1ef733ede6bce6c9a WatchSource:0}: Error finding container 3827329b4707c6d72fe10730b0783ae9f8aba3771ab841d1ef733ede6bce6c9a: Status 404 returned error can't find the container with id 3827329b4707c6d72fe10730b0783ae9f8aba3771ab841d1ef733ede6bce6c9a
Jul 10 14:36:19.233555 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.232885    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645" event={"ID":"fcd96d02-8d9c-436e-aa44-a7a06b4a049d","Type":"ContainerStarted","Data":"60110cd503a9acdc3d52c3215c1d44b34ba73f5bfa328f666e6f3fb6efaf368d"}
Jul 10 14:36:19.233848 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.233793    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:19.235301 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.235224    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfr2g" event={"ID":"657b971b-c38c-45c7-a2e8-f47c4c2f0f68","Type":"ContainerStarted","Data":"3827329b4707c6d72fe10730b0783ae9f8aba3771ab841d1ef733ede6bce6c9a"}
Jul 10 14:36:19.238782 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.238756    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g" event={"ID":"1f887ff3-e876-4c34-a423-949ac8e56f9c","Type":"ContainerStarted","Data":"4c838d89ee6c60c11134fab1397014475165cf2572109f18cc122fc5b2d8ed36"}
Jul 10 14:36:19.239416 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.239370    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:19.241503 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.241202    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645"
Jul 10 14:36:19.241503 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.241356    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g"
Jul 10 14:36:19.242675 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.242294    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt" event={"ID":"7a2eea60-9055-4536-a8d3-a3f90e4915d1","Type":"ContainerStarted","Data":"5a1c7871034711342db1fe138ff008cd24490e4bfc4abe7fe81cdb8620b4f1d9"}
Jul 10 14:36:19.275129 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.274489    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nv645" podStartSLOduration=2.732601369 podStartE2EDuration="7.274468119s" totalImagesPullingTime="4.54186675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:19.273215104 +0000 UTC m=+76.979536574" watchObservedRunningTime="2026-07-10 14:36:19.274468119 +0000 UTC m=+76.980789590"
Jul 10 14:36:19.300850 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.299647    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c85879b4-2g2mt" podStartSLOduration=3.532597028 podStartE2EDuration="8.299626901s" totalImagesPullingTime="4.767029873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:19.298666298 +0000 UTC m=+77.004987768" watchObservedRunningTime="2026-07-10 14:36:19.299626901 +0000 UTC m=+77.005948373"
Jul 10 14:36:19.320143 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.320086    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c665b5c75-lqt9g" podStartSLOduration=3.761538548 podStartE2EDuration="8.320066725s" totalImagesPullingTime="4.558528177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:19.319588566 +0000 UTC m=+77.025910036" watchObservedRunningTime="2026-07-10 14:36:19.320066725 +0000 UTC m=+77.026388195"
Jul 10 14:36:19.638041 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.637943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:19.680611 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.680165    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-l87jd"]
Jul 10 14:36:19.682687 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.682666    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.693929 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.693055    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 14:36:19.693929 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.693374    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 14:36:19.697879 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.697708    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mhhj8\""
Jul 10 14:36:19.698141 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.698122    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 14:36:19.699879 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.699773    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 14:36:19.700631 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.700393    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 14:36:19.708163 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.708120    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-l87jd"]
Jul 10 14:36:19.754221 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.754186    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.754418 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.754241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.754418 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.754305    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96dfh\" (UniqueName: \"kubernetes.io/projected/c0028d35-0a41-4d17-a319-ed2d4cee9168-kube-api-access-96dfh\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.754418 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.754331    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0028d35-0a41-4d17-a319-ed2d4cee9168-metrics-client-ca\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.760332 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.760246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:19.856543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.855188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96dfh\" (UniqueName: \"kubernetes.io/projected/c0028d35-0a41-4d17-a319-ed2d4cee9168-kube-api-access-96dfh\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.856543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.855233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0028d35-0a41-4d17-a319-ed2d4cee9168-metrics-client-ca\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.856543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.855281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.856543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.855310    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.856543 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:19.855461    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 10 14:36:19.856543 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:19.855546    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-tls podName:c0028d35-0a41-4d17-a319-ed2d4cee9168 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:20.35550268 +0000 UTC m=+78.061824133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-tls") pod "prometheus-operator-64785b6675-l87jd" (UID: "c0028d35-0a41-4d17-a319-ed2d4cee9168") : secret "prometheus-operator-tls" not found
Jul 10 14:36:19.857618 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.857573    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0028d35-0a41-4d17-a319-ed2d4cee9168-metrics-client-ca\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.861158 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.861131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:19.865885 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:19.865859    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96dfh\" (UniqueName: \"kubernetes.io/projected/c0028d35-0a41-4d17-a319-ed2d4cee9168-kube-api-access-96dfh\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:20.325386 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:20.325350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:20.359657 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:20.359617    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:20.362268 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:20.362237    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0028d35-0a41-4d17-a319-ed2d4cee9168-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l87jd\" (UID: \"c0028d35-0a41-4d17-a319-ed2d4cee9168\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:20.427165 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:20.427118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:20.598492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:20.597821    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd"
Jul 10 14:36:23.063601 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:23.063571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-l87jd"]
Jul 10 14:36:23.261309 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:23.261217    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfr2g" event={"ID":"657b971b-c38c-45c7-a2e8-f47c4c2f0f68","Type":"ContainerStarted","Data":"3a82d82ddc652ded0fe86fbdc73d5c50527590dbf7e675fe26db8149e8861e99"}
Jul 10 14:36:23.262572 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:23.262543    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd" event={"ID":"c0028d35-0a41-4d17-a319-ed2d4cee9168","Type":"ContainerStarted","Data":"b5422e3becd412286f48d4df02c718b8ab345ba6213d9b966c6b98504dc494b7"}
Jul 10 14:36:24.217006 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:24.216604    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vdw48"
Jul 10 14:36:24.272670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:24.272631    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfr2g" event={"ID":"657b971b-c38c-45c7-a2e8-f47c4c2f0f68","Type":"ContainerStarted","Data":"e4bd330abb1727afa50dcbef810273c8d3f6cbdcf9646e774a567f2d7dcd555a"}
Jul 10 14:36:24.372196 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:24.372161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:24.525044 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:24.524981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:24.880417 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:24.880316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:25.279140 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:25.279105    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd" event={"ID":"c0028d35-0a41-4d17-a319-ed2d4cee9168","Type":"ContainerStarted","Data":"8f6cc74e1fc7b7a8c651edf9e9d15b847ea48f03f208f8771ba29ea1f016719b"}
Jul 10 14:36:25.279615 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:25.279171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd" event={"ID":"c0028d35-0a41-4d17-a319-ed2d4cee9168","Type":"ContainerStarted","Data":"646625b61df7a998b13c08f476175c4314d9d5c496b0b5f288fe37488a96002e"}
Jul 10 14:36:25.300676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:25.300629    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-l87jd" podStartSLOduration=4.86152486 podStartE2EDuration="6.300611637s" totalImagesPullingTime="1.439086777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:25.299690812 +0000 UTC m=+83.006012282" watchObservedRunningTime="2026-07-10 14:36:25.300611637 +0000 UTC m=+83.006933106"
Jul 10 14:36:25.629776 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:25.629678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:25.709470 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:25.709419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:26.849151 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:26.849110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:27.105689 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.105556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:27.150662 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.150631    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"]
Jul 10 14:36:27.153917 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.153896    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.156958 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.156932    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 10 14:36:27.157060 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.156983    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 10 14:36:27.165057 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.165035    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tpc7v\""
Jul 10 14:36:27.199684 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.199656    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"]
Jul 10 14:36:27.217535 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.217489    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.217688 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.217660    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdqf\" (UniqueName: \"kubernetes.io/projected/ab466a2b-183b-42e0-b372-edc6982a946d-kube-api-access-cwdqf\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.217895 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.217701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab466a2b-183b-42e0-b372-edc6982a946d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.217895 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.217740    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.220475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.220453    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n4sc5"]
Jul 10 14:36:27.223922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.223905    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.229171 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.229144    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 14:36:27.230209 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.230147    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 14:36:27.231389 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.231366    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sbjmx\""
Jul 10 14:36:27.232116 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.232090    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 14:36:27.308405 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.308364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:27.318795 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.318763    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-root\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.318940 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.318808    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.318940 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.318842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-textfile\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.318940 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.318891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-metrics-client-ca\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.319065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.318971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdqf\" (UniqueName: \"kubernetes.io/projected/ab466a2b-183b-42e0-b372-edc6982a946d-kube-api-access-cwdqf\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.319065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.318998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8znj7\" (UniqueName: \"kubernetes.io/projected/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-kube-api-access-8znj7\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.319065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab466a2b-183b-42e0-b372-edc6982a946d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.319065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319055    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.319245 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319081    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-accelerators-collector-config\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.319245 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319150    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-sys\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.319245 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.319245 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319235    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-wtmp\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.319434 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:27.319333    2583 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 10 14:36:27.319434 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:27.319396    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-tls podName:ab466a2b-183b-42e0-b372-edc6982a946d nodeName:}" failed. No retries permitted until 2026-07-10 14:36:27.81937624 +0000 UTC m=+85.525697687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-wmr2x" (UID: "ab466a2b-183b-42e0-b372-edc6982a946d") : secret "openshift-state-metrics-tls" not found
Jul 10 14:36:27.319434 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319327    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-tls\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.319868 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.319845    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab466a2b-183b-42e0-b372-edc6982a946d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.321914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.321890    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.352170 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.352148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdqf\" (UniqueName: \"kubernetes.io/projected/ab466a2b-183b-42e0-b372-edc6982a946d-kube-api-access-cwdqf\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.419902 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.419865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znj7\" (UniqueName: \"kubernetes.io/projected/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-kube-api-access-8znj7\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.419915    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-accelerators-collector-config\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.419954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-sys\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-wtmp\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420034    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-tls\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420065    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-root\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-sys\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420130    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-root\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420216    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-wtmp\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-textfile\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-metrics-client-ca\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420575    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-textfile\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420658    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-accelerators-collector-config\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.420880 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.420848    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-metrics-client-ca\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.422760 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.422733    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.422951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.422919    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-node-exporter-tls\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.431117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.431083    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8znj7\" (UniqueName: \"kubernetes.io/projected/ae37bf2b-3bb1-4fe8-94ef-8651a17db324-kube-api-access-8znj7\") pod \"node-exporter-n4sc5\" (UID: \"ae37bf2b-3bb1-4fe8-94ef-8651a17db324\") " pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.437202 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.437175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:27.534914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.534887    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n4sc5"
Jul 10 14:36:27.537365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.537309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:27.748825 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.748738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:27.824235 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.824202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:27.826655 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:27.826633    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab466a2b-183b-42e0-b372-edc6982a946d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wmr2x\" (UID: \"ab466a2b-183b-42e0-b372-edc6982a946d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:28.034764 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.034679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:28.063653 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.063615    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"
Jul 10 14:36:28.145956 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.145918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:28.243946 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.243911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:28.352079 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.352011    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:36:28.355786 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.355762    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.362801 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.362712    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 14:36:28.362801 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.362790    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 14:36:28.362976 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.362806    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 14:36:28.363719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363475    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gcln5\""
Jul 10 14:36:28.363719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363490    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 14:36:28.363719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363580    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 14:36:28.363719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363628    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 14:36:28.363932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363745    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 14:36:28.363932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363859    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 14:36:28.363932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.363873    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 14:36:28.370889 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.370866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:28.384467 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.384445    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:36:28.429849 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.429821    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430031 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.429898    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430031 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.429929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430031 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.429949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430031 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.429973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430031 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.429999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-web-config\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-out\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430144    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.430272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.430218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvqqd\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-kube-api-access-jvqqd\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531159 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531357 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531357 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531357 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531550 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531550 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvqqd\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-kube-api-access-jvqqd\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531550 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531550 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-web-config\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.531749 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.531659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-out\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.532502 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:28.532046    2583 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 10 14:36:28.532502 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:28.532126    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls podName:805d7e89-4c56-4b9d-8bc6-ec4a37a668fb nodeName:}" failed. No retries permitted until 2026-07-10 14:36:29.032104009 +0000 UTC m=+86.738425467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb") : secret "alertmanager-main-tls" not found
Jul 10 14:36:28.532502 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:28.532168    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle podName:805d7e89-4c56-4b9d-8bc6-ec4a37a668fb nodeName:}" failed. No retries permitted until 2026-07-10 14:36:29.032149446 +0000 UTC m=+86.738470910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb") : configmap references non-existent config key: ca-bundle.crt
Jul 10 14:36:28.532502 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.532303    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.534454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.534428    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-out\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.534572 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.534501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.534992 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.534951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.535343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.535319    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.535442 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.535420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.535503 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.535458    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.535601 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.535497    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.536287 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.536269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-web-config\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.542194 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.542176    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvqqd\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-kube-api-access-jvqqd\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:28.685955 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:28.685919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:29.036661 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.036580    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:29.037072 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.036668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:29.037489 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.037465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:29.039165 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.039145    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:29.265888 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.265848    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:36:29.873834 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.873800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:29.975142 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:29.975106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:30.135475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:30.135395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:30.252533 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:30.252480    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:30.428813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:30.428773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:30.572461 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:30.572423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:31.235362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.235318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:31.362232 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.362193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:31.877629 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.877598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:31.906990 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.906956    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"]
Jul 10 14:36:31.957230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.957191    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"]
Jul 10 14:36:31.957403 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.957311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:31.960789 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.960760    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-smxkd\""
Jul 10 14:36:31.960931 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:31.960766    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 14:36:32.006838 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.006778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:32.062390 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.062357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06148697-3ebc-41b0-a129-6e073dfa1bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.062549 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.062401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pp4h\" (UniqueName: \"kubernetes.io/projected/06148697-3ebc-41b0-a129-6e073dfa1bb3-kube-api-access-7pp4h\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.163235 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.163197    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06148697-3ebc-41b0-a129-6e073dfa1bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.163409 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.163249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pp4h\" (UniqueName: \"kubernetes.io/projected/06148697-3ebc-41b0-a129-6e073dfa1bb3-kube-api-access-7pp4h\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.163409 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:32.163369    2583 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 10 14:36:32.163510 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:32.163455    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06148697-3ebc-41b0-a129-6e073dfa1bb3-monitoring-plugin-cert podName:06148697-3ebc-41b0-a129-6e073dfa1bb3 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:32.663432253 +0000 UTC m=+90.369753702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/06148697-3ebc-41b0-a129-6e073dfa1bb3-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-chfpp" (UID: "06148697-3ebc-41b0-a129-6e073dfa1bb3") : secret "monitoring-plugin-cert" not found
Jul 10 14:36:32.178509 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.178482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pp4h\" (UniqueName: \"kubernetes.io/projected/06148697-3ebc-41b0-a129-6e073dfa1bb3-kube-api-access-7pp4h\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.191949 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.191924    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:32.382963 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.382926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:32.667616 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.667577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06148697-3ebc-41b0-a129-6e073dfa1bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.670274 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.670249    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06148697-3ebc-41b0-a129-6e073dfa1bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-chfpp\" (UID: \"06148697-3ebc-41b0-a129-6e073dfa1bb3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:32.868658 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:32.868632    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:33.402783 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.402705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:33.476795 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.476755    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:36:33.511377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.511286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:33.526001 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.525370    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:36:33.526001 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.525563    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.528750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.528724    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 14:36:33.528910 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.528783    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mf8t7\""
Jul 10 14:36:33.528910 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.528840    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 14:36:33.529038 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.528919    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 14:36:33.529038 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.528972    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 14:36:33.529214 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.529200    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 14:36:33.531731 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.531696    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 14:36:33.532872 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.532825    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 14:36:33.533019 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.532927    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 14:36:33.533225 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.533203    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 14:36:33.533345 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.533334    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 14:36:33.533407 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.533362    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 14:36:33.536343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.536322    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 14:36:33.537058 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.536776    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 14:36:33.538180 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.537829    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f5jnnl3o8tnje\""
Jul 10 14:36:33.546420 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.546398    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 14:36:33.547933 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.547914    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 14:36:33.613331 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.613302    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:33.676777 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676740    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.676928 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676787    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.676928 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676847    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.676928 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676886    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvwc\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-kube-api-access-cfvwc\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677089 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677089 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677089 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.676976    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677089 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677089 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-web-config\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677096    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-config-out\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677148    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677180    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-config\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677230    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677255    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677307    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677608 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677333    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677608 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.677608 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.677364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.716862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.716831    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:33.778587 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778553    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778587 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778592    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778826 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778840    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-web-config\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778859    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-config-out\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778949 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778874    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778949 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.778949 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-config\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.778979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779006    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779047    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779074    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779096    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779558 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779558 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779211    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779558 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779558 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvwc\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-kube-api-access-cfvwc\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779558 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779438    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779558 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779874 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.779933 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.779888    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.780457 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.780429    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.781835 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.781808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.782074 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.782050    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-config-out\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.782445 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.782262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.783886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.782587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-web-config\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.783886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.782729    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.783886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.783160    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.783886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.783173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.783886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.783307    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.783886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.783758    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.784234 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.784210    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.784234 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.784223    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.784664 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.784639    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-config\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.784748 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.784687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.785255 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.785237    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.788110 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.788091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvwc\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-kube-api-access-cfvwc\") pod \"prometheus-k8s-0\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.841674 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:33.841562    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:33.932769 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:33.932736    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae37bf2b_3bb1_4fe8_94ef_8651a17db324.slice/crio-ee0cb05334375aaf5af0dfa491b85cb8c32eeeff1d6f54efa53753af9331f1dd WatchSource:0}: Error finding container ee0cb05334375aaf5af0dfa491b85cb8c32eeeff1d6f54efa53753af9331f1dd: Status 404 returned error can't find the container with id ee0cb05334375aaf5af0dfa491b85cb8c32eeeff1d6f54efa53753af9331f1dd
Jul 10 14:36:34.102850 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.102818    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x"]
Jul 10 14:36:34.104952 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:34.104927    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab466a2b_183b_42e0_b372_edc6982a946d.slice/crio-c7314688877426b4d0ea24574065f410cf60eda366a1515541fcbc021b37a927 WatchSource:0}: Error finding container c7314688877426b4d0ea24574065f410cf60eda366a1515541fcbc021b37a927: Status 404 returned error can't find the container with id c7314688877426b4d0ea24574065f410cf60eda366a1515541fcbc021b37a927
Jul 10 14:36:34.141411 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.141385    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:36:34.143504 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:34.143481    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ab2a6c_ec16_4254_9bb5_276d53845c61.slice/crio-3ed05fdbd925cbace37ed858e8cbbad2c519d1379a924a6450a9042575da5812 WatchSource:0}: Error finding container 3ed05fdbd925cbace37ed858e8cbbad2c519d1379a924a6450a9042575da5812: Status 404 returned error can't find the container with id 3ed05fdbd925cbace37ed858e8cbbad2c519d1379a924a6450a9042575da5812
Jul 10 14:36:34.320565 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.320072    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"3ed05fdbd925cbace37ed858e8cbbad2c519d1379a924a6450a9042575da5812"}
Jul 10 14:36:34.321974 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.321947    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x" event={"ID":"ab466a2b-183b-42e0-b372-edc6982a946d","Type":"ContainerStarted","Data":"70cb08fa4ead31836f0337cfb02878e6d3afbbd2218b57784a506bff43d0893d"}
Jul 10 14:36:34.322102 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.322002    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x" event={"ID":"ab466a2b-183b-42e0-b372-edc6982a946d","Type":"ContainerStarted","Data":"17a4aea90088183f6bd1e4de97e8d13275703ff121eca05e4c31ce0710023431"}
Jul 10 14:36:34.322102 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.322030    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x" event={"ID":"ab466a2b-183b-42e0-b372-edc6982a946d","Type":"ContainerStarted","Data":"c7314688877426b4d0ea24574065f410cf60eda366a1515541fcbc021b37a927"}
Jul 10 14:36:34.323378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.323355    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n4sc5" event={"ID":"ae37bf2b-3bb1-4fe8-94ef-8651a17db324","Type":"ContainerStarted","Data":"ee0cb05334375aaf5af0dfa491b85cb8c32eeeff1d6f54efa53753af9331f1dd"}
Jul 10 14:36:34.325263 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.324982    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-wldc5" event={"ID":"801e38db-10ba-44b5-90c4-bcce65f80b3f","Type":"ContainerStarted","Data":"2d489f613d824639958ef4fb851e3f2f44e6f633f66dd447d8a9687845d33b2e"}
Jul 10 14:36:34.325263 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.325199    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:34.326868 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.326811    2583 patch_prober.go:28] interesting pod/downloads-68c9cbb497-wldc5 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" start-of-body=
Jul 10 14:36:34.326963 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.326854    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-wldc5" podUID="801e38db-10ba-44b5-90c4-bcce65f80b3f" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused"
Jul 10 14:36:34.327221 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.327190    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rfr2g" event={"ID":"657b971b-c38c-45c7-a2e8-f47c4c2f0f68","Type":"ContainerStarted","Data":"307f3c11bbb5e62676b05156ffd0f8c2c983e59b91cbc4efda70b6df3532c6fa"}
Jul 10 14:36:34.349445 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.349399    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-wldc5" podStartSLOduration=2.9789443799999997 podStartE2EDuration="23.349384225s" totalImagesPullingTime="20.370439845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:34.347675464 +0000 UTC m=+92.053996944" watchObservedRunningTime="2026-07-10 14:36:34.349384225 +0000 UTC m=+92.055705672"
Jul 10 14:36:34.355724 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.355701    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"]
Jul 10 14:36:34.359258 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:34.359229    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06148697_3ebc_41b0_a129_6e073dfa1bb3.slice/crio-fa8bac5ed52d23da8095691561bba637dc0e25b997305c12855661eb0e8e09f6 WatchSource:0}: Error finding container fa8bac5ed52d23da8095691561bba637dc0e25b997305c12855661eb0e8e09f6: Status 404 returned error can't find the container with id fa8bac5ed52d23da8095691561bba637dc0e25b997305c12855661eb0e8e09f6
Jul 10 14:36:34.361266 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.361243    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:36:34.363686 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:36:34.363658    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805d7e89_4c56_4b9d_8bc6_ec4a37a668fb.slice/crio-f504ca29e66b715d9b44e185cd901faec7aeffd425b2a1de807ecb293318b4af WatchSource:0}: Error finding container f504ca29e66b715d9b44e185cd901faec7aeffd425b2a1de807ecb293318b4af: Status 404 returned error can't find the container with id f504ca29e66b715d9b44e185cd901faec7aeffd425b2a1de807ecb293318b4af
Jul 10 14:36:34.381842 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.381786    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rfr2g" podStartSLOduration=33.805651001 podStartE2EDuration="49.381766148s" totalImagesPullingTime="15.576115147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:34.380687334 +0000 UTC m=+92.087008804" watchObservedRunningTime="2026-07-10 14:36:34.381766148 +0000 UTC m=+92.088087617"
Jul 10 14:36:34.780053 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.780017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:34.884317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:34.884285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:35.232710 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:35.232668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:35.306034 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:35.305959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:35.334580 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:35.334434    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp" event={"ID":"06148697-3ebc-41b0-a129-6e073dfa1bb3","Type":"ContainerStarted","Data":"fa8bac5ed52d23da8095691561bba637dc0e25b997305c12855661eb0e8e09f6"}
Jul 10 14:36:35.340414 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:35.338757    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"f504ca29e66b715d9b44e185cd901faec7aeffd425b2a1de807ecb293318b4af"}
Jul 10 14:36:35.352468 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:35.352210    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-wldc5"
Jul 10 14:36:37.207867 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.207821    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-559554586f-bgs9g" podUID="ddc0763c-c27d-485d-a69b-c8187a209a09" containerName="registry" containerID="cri-o://3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e" gracePeriod=30
Jul 10 14:36:37.549991 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.549962    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:37.726146 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726117    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-trusted-ca\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726262 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726177    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726262 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726223    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-certificates\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726369 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726262    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-bound-sa-token\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726369 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726309    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-installation-pull-secrets\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726369 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726340    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddc0763c-c27d-485d-a69b-c8187a209a09-ca-trust-extracted\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726534 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726404    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-image-registry-private-configuration\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726534 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726454    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cms5v\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-kube-api-access-cms5v\") pod \"ddc0763c-c27d-485d-a69b-c8187a209a09\" (UID: \"ddc0763c-c27d-485d-a69b-c8187a209a09\") "
Jul 10 14:36:37.726643 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726534    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:36:37.727024 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.726798    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-trusted-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.727732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.727704    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:36:37.733665 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.733635    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:36:37.734544 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.733819    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:36:37.734544 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.733916    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-kube-api-access-cms5v" (OuterVolumeSpecName: "kube-api-access-cms5v") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "kube-api-access-cms5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:36:37.734544 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.734132    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:36:37.734991 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.734952    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:36:37.744042 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.744010    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc0763c-c27d-485d-a69b-c8187a209a09-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ddc0763c-c27d-485d-a69b-c8187a209a09" (UID: "ddc0763c-c27d-485d-a69b-c8187a209a09"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828198    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cms5v\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-kube-api-access-cms5v\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828229    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828242    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddc0763c-c27d-485d-a69b-c8187a209a09-registry-certificates\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828255    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddc0763c-c27d-485d-a69b-c8187a209a09-bound-sa-token\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828267    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-installation-pull-secrets\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828282    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddc0763c-c27d-485d-a69b-c8187a209a09-ca-trust-extracted\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:37.828315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:37.828294    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddc0763c-c27d-485d-a69b-c8187a209a09-image-registry-private-configuration\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:36:38.360106 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.360065    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84" exitCode=0
Jul 10 14:36:38.360578 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.360152    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"}
Jul 10 14:36:38.362822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.362792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x" event={"ID":"ab466a2b-183b-42e0-b372-edc6982a946d","Type":"ContainerStarted","Data":"a9a0640296bbfde3d751088e9917120b105814b12cc21797349cd05679597504"}
Jul 10 14:36:38.364796 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.364767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp" event={"ID":"06148697-3ebc-41b0-a129-6e073dfa1bb3","Type":"ContainerStarted","Data":"7b63820520669df52e2696e828cebd53b9bae9f66fe8368be0733b1fe0e70e1c"}
Jul 10 14:36:38.365178 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.365157    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:38.367623 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.367596    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0" exitCode=0
Jul 10 14:36:38.367734 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.367631    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"}
Jul 10 14:36:38.369681 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.369628    2583 generic.go:356] "Generic (PLEG): container finished" podID="ddc0763c-c27d-485d-a69b-c8187a209a09" containerID="3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e" exitCode=0
Jul 10 14:36:38.369743 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.369681    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559554586f-bgs9g"
Jul 10 14:36:38.369743 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.369707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559554586f-bgs9g" event={"ID":"ddc0763c-c27d-485d-a69b-c8187a209a09","Type":"ContainerDied","Data":"3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e"}
Jul 10 14:36:38.369854 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.369744    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559554586f-bgs9g" event={"ID":"ddc0763c-c27d-485d-a69b-c8187a209a09","Type":"ContainerDied","Data":"fc90ba8dbbc48e8e143921a93d8238711bb1387cceee6a3400f83904551f826b"}
Jul 10 14:36:38.369854 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.369761    2583 scope.go:117] "RemoveContainer" containerID="3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e"
Jul 10 14:36:38.371497 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.371456    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp"
Jul 10 14:36:38.372133 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.372111    2583 generic.go:356] "Generic (PLEG): container finished" podID="ae37bf2b-3bb1-4fe8-94ef-8651a17db324" containerID="c599307bf02a8092ddb884e2515306ce0ed1f5e6a34c7059ff1b60afc717eeb5" exitCode=0
Jul 10 14:36:38.372242 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.372161    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n4sc5" event={"ID":"ae37bf2b-3bb1-4fe8-94ef-8651a17db324","Type":"ContainerDied","Data":"c599307bf02a8092ddb884e2515306ce0ed1f5e6a34c7059ff1b60afc717eeb5"}
Jul 10 14:36:38.384151 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.384134    2583 scope.go:117] "RemoveContainer" containerID="3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e"
Jul 10 14:36:38.384466 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:36:38.384443    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e\": container with ID starting with 3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e not found: ID does not exist" containerID="3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e"
Jul 10 14:36:38.384550 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.384478    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e"} err="failed to get container status \"3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e\": rpc error: code = NotFound desc = could not find container \"3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e\": container with ID starting with 3f27e9824b0a0184f35209eba04708829e12c2d553dc45777ed002857901d62e not found: ID does not exist"
Jul 10 14:36:38.430412 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.430382    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559554586f-bgs9g"]
Jul 10 14:36:38.444408 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.444384    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-559554586f-bgs9g"]
Jul 10 14:36:38.474449 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.474400    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-chfpp" podStartSLOduration=4.115075305 podStartE2EDuration="7.474382464s" totalImagesPullingTime="3.359307159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:38.473161857 +0000 UTC m=+96.179483326" watchObservedRunningTime="2026-07-10 14:36:38.474382464 +0000 UTC m=+96.180703924"
Jul 10 14:36:38.566878 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.566830    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wmr2x" podStartSLOduration=8.131376239 podStartE2EDuration="11.566809263s" totalImagesPullingTime="3.435433024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:38.565780255 +0000 UTC m=+96.272101726" watchObservedRunningTime="2026-07-10 14:36:38.566809263 +0000 UTC m=+96.273130733"
Jul 10 14:36:38.846429 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:38.846386    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc0763c-c27d-485d-a69b-c8187a209a09" path="/var/lib/kubelet/pods/ddc0763c-c27d-485d-a69b-c8187a209a09/volumes"
Jul 10 14:36:39.183279 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:39.183188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:39.285951 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:39.285812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:39.390757 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:39.389966    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n4sc5" event={"ID":"ae37bf2b-3bb1-4fe8-94ef-8651a17db324","Type":"ContainerStarted","Data":"5792cefac0001f63e9fe926913a981e0feae3705f7aedf10509eecfb96b8dae7"}
Jul 10 14:36:39.390757 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:39.390021    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n4sc5" event={"ID":"ae37bf2b-3bb1-4fe8-94ef-8651a17db324","Type":"ContainerStarted","Data":"cd12ecbfd1954fc08a3b6187b57d601b53820822d2e6bc6356624704f0ae9cf3"}
Jul 10 14:36:39.422184 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:39.421088    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n4sc5" podStartSLOduration=8.82438617 podStartE2EDuration="12.421064792s" totalImagesPullingTime="3.478042305s" totalInitContainerRuntime="118.636317ms" isStatefulPod=true podCreationTimestamp="2026-07-10 14:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:39.418868387 +0000 UTC m=+97.125189856" watchObservedRunningTime="2026-07-10 14:36:39.421064792 +0000 UTC m=+97.127386262"
Jul 10 14:36:40.332802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:40.332771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:40.438262 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:40.438200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:43.413500 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:43.413456    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"}
Jul 10 14:36:43.414025 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:43.413572    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"}
Jul 10 14:36:43.418378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:43.417980    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"}
Jul 10 14:36:43.418378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:43.418077    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"}
Jul 10 14:36:43.418378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:43.418140    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"}
Jul 10 14:36:43.418378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:43.418208    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"}
Jul 10 14:36:44.333670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:44.333639    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:44.412213 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:44.412175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:44.426178 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:44.426133    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"}
Jul 10 14:36:45.216572 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:45.216542    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b58rl"
Jul 10 14:36:45.435721 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:45.435687    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerStarted","Data":"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"}
Jul 10 14:36:45.465719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:45.465657    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.842428354 podStartE2EDuration="17.465636084s" totalImagesPullingTime="7.602562632s" totalInitContainerRuntime="20.645098ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:45.464885966 +0000 UTC m=+103.171207469" watchObservedRunningTime="2026-07-10 14:36:45.465636084 +0000 UTC m=+103.171957554"
Jul 10 14:36:46.441965 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:46.441934    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"}
Jul 10 14:36:46.442338 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:46.442026    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"}
Jul 10 14:36:46.442338 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:46.442069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"}
Jul 10 14:36:47.449063 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:47.449029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerStarted","Data":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"}
Jul 10 14:36:47.487144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:47.487081    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.467138615 podStartE2EDuration="14.487067072s" totalImagesPullingTime="10.980233592s" totalInitContainerRuntime="39.694865ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:47.485726968 +0000 UTC m=+105.192048436" watchObservedRunningTime="2026-07-10 14:36:47.487067072 +0000 UTC m=+105.193388541"
Jul 10 14:36:48.840915 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:48.840877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:48.844396 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:48.844375    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:36:48.923237 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:48.923184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:49.024564 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:49.024537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:55.412018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:55.411961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:55.479827 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:36:55.479779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:02.356708 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:02.356674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:10.327202 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:10.327159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:10.392972 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:10.392930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:18.688132 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:18.688085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:33.842325 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:33.842279    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:33.861156 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:33.861124    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:34.648323 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:34.648295    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:36.913175 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:36.913126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:43.339230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:43.339196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:43.439912 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:43.439877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:47.706736 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.706708    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:37:47.707117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.706995    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="alertmanager" containerID="cri-o://83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7" gracePeriod=120
Jul 10 14:37:47.707117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.707075    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy" containerID="cri-o://71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b" gracePeriod=120
Jul 10 14:37:47.707117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.707063    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-metric" containerID="cri-o://b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f" gracePeriod=120
Jul 10 14:37:47.707117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.707103    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="config-reloader" containerID="cri-o://b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae" gracePeriod=120
Jul 10 14:37:47.707331 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.707117    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="prom-label-proxy" containerID="cri-o://733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97" gracePeriod=120
Jul 10 14:37:47.707331 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:47.707092    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-web" containerID="cri-o://8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5" gracePeriod=120
Jul 10 14:37:48.684389 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684355    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97" exitCode=0
Jul 10 14:37:48.684389 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684382    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b" exitCode=0
Jul 10 14:37:48.684389 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684389    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae" exitCode=0
Jul 10 14:37:48.684389 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684396    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7" exitCode=0
Jul 10 14:37:48.684676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684431    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"}
Jul 10 14:37:48.684676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"}
Jul 10 14:37:48.684676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"}
Jul 10 14:37:48.684676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.684589    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"}
Jul 10 14:37:48.942539 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:48.942465    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.022074 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022046    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-tls-assets\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022074 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022076    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvqqd\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-kube-api-access-jvqqd\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022298 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022101    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022298 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022119    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022298 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022149    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-main-db\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022311    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-web-config\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022364    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022392    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-metrics-client-ca\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022448    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-volume\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022704 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-out\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022704 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022472    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:37:49.022704 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022704 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022588    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022704 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022614    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-cluster-tls-config\") pod \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\" (UID: \"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb\") "
Jul 10 14:37:49.022952 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.022863    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-main-db\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.024139 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.024107    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:49.024756 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.024719    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:49.024891 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.024867    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.025088 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.025065    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.025624 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.025592    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.026454 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.026420    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:37:49.026764 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.026728    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-volume" (OuterVolumeSpecName: "config-volume") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.026853 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.026768    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-out" (OuterVolumeSpecName: "config-out") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:37:49.026912 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.026888    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-kube-api-access-jvqqd" (OuterVolumeSpecName: "kube-api-access-jvqqd") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "kube-api-access-jvqqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:37:49.027875 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.027843    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.030866 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.030843    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.036555 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.036535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-web-config" (OuterVolumeSpecName: "web-config") pod "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" (UID: "805d7e89-4c56-4b9d-8bc6-ec4a37a668fb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:49.123365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123337    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-web-config\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123364    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123376    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-metrics-client-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123384    2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-volume\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123392    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-config-out\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123401    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-main-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123412    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123421    2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-cluster-tls-config\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123429    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-tls-assets\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123437    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvqqd\" (UniqueName: \"kubernetes.io/projected/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-kube-api-access-jvqqd\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123445    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.123563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.123453    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:49.691446 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691410    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f" exitCode=0
Jul 10 14:37:49.691446 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691444    2583 generic.go:356] "Generic (PLEG): container finished" podID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerID="8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5" exitCode=0
Jul 10 14:37:49.691671 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691502    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.691671 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691508    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"}
Jul 10 14:37:49.691671 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"}
Jul 10 14:37:49.691671 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691627    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"805d7e89-4c56-4b9d-8bc6-ec4a37a668fb","Type":"ContainerDied","Data":"f504ca29e66b715d9b44e185cd901faec7aeffd425b2a1de807ecb293318b4af"}
Jul 10 14:37:49.691671 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.691658    2583 scope.go:117] "RemoveContainer" containerID="733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"
Jul 10 14:37:49.701927 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.701909    2583 scope.go:117] "RemoveContainer" containerID="b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"
Jul 10 14:37:49.708777 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.708761    2583 scope.go:117] "RemoveContainer" containerID="71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"
Jul 10 14:37:49.715195 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.715177    2583 scope.go:117] "RemoveContainer" containerID="8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"
Jul 10 14:37:49.717946 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.717818    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:37:49.722161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.722138    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:37:49.723017 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.723000    2583 scope.go:117] "RemoveContainer" containerID="b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"
Jul 10 14:37:49.729529 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.729495    2583 scope.go:117] "RemoveContainer" containerID="83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"
Jul 10 14:37:49.736074 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.736059    2583 scope.go:117] "RemoveContainer" containerID="6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"
Jul 10 14:37:49.742721 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.742704    2583 scope.go:117] "RemoveContainer" containerID="733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"
Jul 10 14:37:49.742967 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.742951    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97\": container with ID starting with 733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97 not found: ID does not exist" containerID="733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"
Jul 10 14:37:49.743010 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.742976    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"} err="failed to get container status \"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97\": rpc error: code = NotFound desc = could not find container \"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97\": container with ID starting with 733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97 not found: ID does not exist"
Jul 10 14:37:49.743010 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.742994    2583 scope.go:117] "RemoveContainer" containerID="b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"
Jul 10 14:37:49.743233 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.743215    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f\": container with ID starting with b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f not found: ID does not exist" containerID="b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"
Jul 10 14:37:49.743281 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743238    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"} err="failed to get container status \"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f\": rpc error: code = NotFound desc = could not find container \"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f\": container with ID starting with b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f not found: ID does not exist"
Jul 10 14:37:49.743281 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743253    2583 scope.go:117] "RemoveContainer" containerID="71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"
Jul 10 14:37:49.743474 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.743456    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b\": container with ID starting with 71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b not found: ID does not exist" containerID="71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"
Jul 10 14:37:49.743525 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743479    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"} err="failed to get container status \"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b\": rpc error: code = NotFound desc = could not find container \"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b\": container with ID starting with 71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b not found: ID does not exist"
Jul 10 14:37:49.743525 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743493    2583 scope.go:117] "RemoveContainer" containerID="8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"
Jul 10 14:37:49.743722 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.743706    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5\": container with ID starting with 8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5 not found: ID does not exist" containerID="8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"
Jul 10 14:37:49.743771 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743725    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"} err="failed to get container status \"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5\": rpc error: code = NotFound desc = could not find container \"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5\": container with ID starting with 8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5 not found: ID does not exist"
Jul 10 14:37:49.743771 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743737    2583 scope.go:117] "RemoveContainer" containerID="b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"
Jul 10 14:37:49.743939 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.743924    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae\": container with ID starting with b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae not found: ID does not exist" containerID="b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"
Jul 10 14:37:49.743977 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743943    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"} err="failed to get container status \"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae\": rpc error: code = NotFound desc = could not find container \"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae\": container with ID starting with b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae not found: ID does not exist"
Jul 10 14:37:49.743977 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.743954    2583 scope.go:117] "RemoveContainer" containerID="83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"
Jul 10 14:37:49.744152 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.744137    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7\": container with ID starting with 83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7 not found: ID does not exist" containerID="83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"
Jul 10 14:37:49.744196 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744154    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"} err="failed to get container status \"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7\": rpc error: code = NotFound desc = could not find container \"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7\": container with ID starting with 83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7 not found: ID does not exist"
Jul 10 14:37:49.744196 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744165    2583 scope.go:117] "RemoveContainer" containerID="6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"
Jul 10 14:37:49.744341 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:49.744326    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0\": container with ID starting with 6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0 not found: ID does not exist" containerID="6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"
Jul 10 14:37:49.744378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744342    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"} err="failed to get container status \"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0\": rpc error: code = NotFound desc = could not find container \"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0\": container with ID starting with 6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0 not found: ID does not exist"
Jul 10 14:37:49.744378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744354    2583 scope.go:117] "RemoveContainer" containerID="733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"
Jul 10 14:37:49.744560 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744539    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97"} err="failed to get container status \"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97\": rpc error: code = NotFound desc = could not find container \"733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97\": container with ID starting with 733c70fff0def4c05b0d2b5964f165f907264df9524b37743699a07ccd8c8c97 not found: ID does not exist"
Jul 10 14:37:49.744610 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744562    2583 scope.go:117] "RemoveContainer" containerID="b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"
Jul 10 14:37:49.744760 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744746    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f"} err="failed to get container status \"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f\": rpc error: code = NotFound desc = could not find container \"b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f\": container with ID starting with b42ef0dde7508f46827c552c6dc0466390bf8ea6fbf9654b0a455f5471d4fe5f not found: ID does not exist"
Jul 10 14:37:49.744801 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744760    2583 scope.go:117] "RemoveContainer" containerID="71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"
Jul 10 14:37:49.744950 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744935    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b"} err="failed to get container status \"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b\": rpc error: code = NotFound desc = could not find container \"71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b\": container with ID starting with 71e823ea91edfcde99e5ea4c35ef1bf261a3f3ac8a249d09abdeb8421f59723b not found: ID does not exist"
Jul 10 14:37:49.744997 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.744950    2583 scope.go:117] "RemoveContainer" containerID="8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"
Jul 10 14:37:49.745162 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745141    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5"} err="failed to get container status \"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5\": rpc error: code = NotFound desc = could not find container \"8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5\": container with ID starting with 8ad77fcf72d7ad02411db68136f0e5b5be4eb03154f8ae12a04f999e754aa7a5 not found: ID does not exist"
Jul 10 14:37:49.745214 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745162    2583 scope.go:117] "RemoveContainer" containerID="b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"
Jul 10 14:37:49.745352 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745336    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae"} err="failed to get container status \"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae\": rpc error: code = NotFound desc = could not find container \"b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae\": container with ID starting with b491de46d723938fa1fd2ed2893f6378996afe467edad6b1cb052ea1a55b18ae not found: ID does not exist"
Jul 10 14:37:49.745391 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745354    2583 scope.go:117] "RemoveContainer" containerID="83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"
Jul 10 14:37:49.745561 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745541    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7"} err="failed to get container status \"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7\": rpc error: code = NotFound desc = could not find container \"83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7\": container with ID starting with 83fbfeb948e9a102ba94eb868680bca9b9a9f29e3232d56923f0e8492f3833a7 not found: ID does not exist"
Jul 10 14:37:49.745611 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745561    2583 scope.go:117] "RemoveContainer" containerID="6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"
Jul 10 14:37:49.745742 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.745723    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0"} err="failed to get container status \"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0\": rpc error: code = NotFound desc = could not find container \"6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0\": container with ID starting with 6a7c64c3faf35528349a365dd0f3bcae473b842c49a213aae1f9343fae7f7be0 not found: ID does not exist"
Jul 10 14:37:49.749482 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749462    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:37:49.749748 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749736    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc0763c-c27d-485d-a69b-c8187a209a09" containerName="registry"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749750    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc0763c-c27d-485d-a69b-c8187a209a09" containerName="registry"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749757    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749763    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749769    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="alertmanager"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749774    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="alertmanager"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749781    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="init-config-reloader"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749786    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="init-config-reloader"
Jul 10 14:37:49.749794 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749793    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-web"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749798    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-web"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749804    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="prom-label-proxy"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749810    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="prom-label-proxy"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749826    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-metric"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749831    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-metric"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749840    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="config-reloader"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749845    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="config-reloader"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749888    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="alertmanager"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749896    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749902    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="prom-label-proxy"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749909    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="config-reloader"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749915    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-web"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749920    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc0763c-c27d-485d-a69b-c8187a209a09" containerName="registry"
Jul 10 14:37:49.750013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.749926    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" containerName="kube-rbac-proxy-metric"
Jul 10 14:37:49.753586 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.753571    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.756596 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756571    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 14:37:49.756699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756604    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 14:37:49.756699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756611    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 14:37:49.756699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756571    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 14:37:49.756699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756686    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gcln5\""
Jul 10 14:37:49.756891 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756717    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 14:37:49.756891 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756728    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 14:37:49.756891 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.756842    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 14:37:49.757691 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.757671    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 14:37:49.763042 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.763026    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 14:37:49.773223 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.773200    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:37:49.828317 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828290    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-config-volume\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a59f7884-efb6-49ec-b636-a0ee313cc909-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828342    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a59f7884-efb6-49ec-b636-a0ee313cc909-config-out\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a59f7884-efb6-49ec-b636-a0ee313cc909-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvll5\" (UniqueName: \"kubernetes.io/projected/a59f7884-efb6-49ec-b636-a0ee313cc909-kube-api-access-jvll5\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828453 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828454    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828568    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a59f7884-efb6-49ec-b636-a0ee313cc909-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828595    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-web-config\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828613    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.828750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.828630    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a59f7884-efb6-49ec-b636-a0ee313cc909-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.929863 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.929831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-config-volume\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.929863 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.929868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a59f7884-efb6-49ec-b636-a0ee313cc909-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.929898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.929951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.929983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a59f7884-efb6-49ec-b636-a0ee313cc909-config-out\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a59f7884-efb6-49ec-b636-a0ee313cc909-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvll5\" (UniqueName: \"kubernetes.io/projected/a59f7884-efb6-49ec-b636-a0ee313cc909-kube-api-access-jvll5\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a59f7884-efb6-49ec-b636-a0ee313cc909-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-web-config\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a59f7884-efb6-49ec-b636-a0ee313cc909-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.930681 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930661    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a59f7884-efb6-49ec-b636-a0ee313cc909-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.931450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.930831    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a59f7884-efb6-49ec-b636-a0ee313cc909-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.932781 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.932756    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a59f7884-efb6-49ec-b636-a0ee313cc909-config-out\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.932869 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.932834    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a59f7884-efb6-49ec-b636-a0ee313cc909-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.933046 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.932985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.933164 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.933051    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-config-volume\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.933286 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.933244    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-web-config\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.933286 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.933271    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.933648 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.933628    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.933752 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.933730    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a59f7884-efb6-49ec-b636-a0ee313cc909-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.934034 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.934014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.934750 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.934732    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a59f7884-efb6-49ec-b636-a0ee313cc909-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:49.940467 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:49.940451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvll5\" (UniqueName: \"kubernetes.io/projected/a59f7884-efb6-49ec-b636-a0ee313cc909-kube-api-access-jvll5\") pod \"alertmanager-main-0\" (UID: \"a59f7884-efb6-49ec-b636-a0ee313cc909\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:50.072536 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:50.072447    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 14:37:50.208752 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:50.208672    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 14:37:50.210829 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:37:50.210804    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59f7884_efb6_49ec_b636_a0ee313cc909.slice/crio-d47a86664282d58850f50beba754d0ffabb6de2d97095e9cb4e42eab5510b11c WatchSource:0}: Error finding container d47a86664282d58850f50beba754d0ffabb6de2d97095e9cb4e42eab5510b11c: Status 404 returned error can't find the container with id d47a86664282d58850f50beba754d0ffabb6de2d97095e9cb4e42eab5510b11c
Jul 10 14:37:50.697230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:50.697196    2583 generic.go:356] "Generic (PLEG): container finished" podID="a59f7884-efb6-49ec-b636-a0ee313cc909" containerID="f36c93028a296e2560a5b42e9714e4914f18f239d1c848a50316b5a5fb84a4e3" exitCode=0
Jul 10 14:37:50.697387 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:50.697250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerDied","Data":"f36c93028a296e2560a5b42e9714e4914f18f239d1c848a50316b5a5fb84a4e3"}
Jul 10 14:37:50.697387 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:50.697356    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"d47a86664282d58850f50beba754d0ffabb6de2d97095e9cb4e42eab5510b11c"}
Jul 10 14:37:50.846205 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:50.846125    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805d7e89-4c56-4b9d-8bc6-ec4a37a668fb" path="/var/lib/kubelet/pods/805d7e89-4c56-4b9d-8bc6-ec4a37a668fb/volumes"
Jul 10 14:37:51.704258 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.704226    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"fd53668fa15dbf3470873e63e99045be58ed234d0777ae9ab88214f681a442f8"}
Jul 10 14:37:51.704639 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.704291    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"28ac261ee21b960527513a4e68448add6c33308f6a998e6313368344bf6f3a05"}
Jul 10 14:37:51.704639 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.704316    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"98f7cf067e0fb91f993a69a945c076bada7098a2f1a2998b02ed6fcb963b6031"}
Jul 10 14:37:51.704639 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.704345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"bec3143d2f8703adf0335c7947f5fb7de82efadcd43b9c8f7f43e06406da99d6"}
Jul 10 14:37:51.704639 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.704368    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"b45a65584d696bf545588049014d1a4f429a9e6c353155877dddd29598ab4fa6"}
Jul 10 14:37:51.704639 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.704391    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a59f7884-efb6-49ec-b636-a0ee313cc909","Type":"ContainerStarted","Data":"f8be0880233f512ae0136d5947698994312b71c2fce6004538e54475d3209a46"}
Jul 10 14:37:51.735674 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.735611    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.721663012 podStartE2EDuration="2.735589482s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.92647ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:37:51.733081905 +0000 UTC m=+169.439403373" watchObservedRunningTime="2026-07-10 14:37:51.735589482 +0000 UTC m=+169.441910955"
Jul 10 14:37:51.974966 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.974883    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:37:51.975382 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.975175    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="prometheus" containerID="cri-o://4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d" gracePeriod=600
Jul 10 14:37:51.975382 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.975200    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="thanos-sidecar" containerID="cri-o://d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e" gracePeriod=600
Jul 10 14:37:51.975382 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.975202    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy" containerID="cri-o://aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c" gracePeriod=600
Jul 10 14:37:51.975382 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.975283    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-web" containerID="cri-o://bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d" gracePeriod=600
Jul 10 14:37:51.975382 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.975285    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="config-reloader" containerID="cri-o://4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500" gracePeriod=600
Jul 10 14:37:51.975382 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:51.975340    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-thanos" containerID="cri-o://18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307" gracePeriod=600
Jul 10 14:37:52.219922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.219898    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.246129 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246057    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfvwc\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-kube-api-access-cfvwc\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246129 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246086    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-2\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246129 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246109    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-grpc-tls\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246129 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246129    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-metrics-client-certs\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246147    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-tls\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-kube-rbac-proxy\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246216    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-kubelet-serving-ca-bundle\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246250    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-db\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246281    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-trusted-ca-bundle\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246305    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-metrics-client-ca\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246334    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-config\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246380    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-serving-certs-ca-bundle\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246419    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-thanos-prometheus-http-client-file\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246435    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246465    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-tls-assets\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246492    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246533    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-0\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246627    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-config-out\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246662    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-web-config\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246689    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-1\") pod \"82ab2a6c-ec16-4254-9bb5-276d53845c61\" (UID: \"82ab2a6c-ec16-4254-9bb5-276d53845c61\") "
Jul 10 14:37:52.246802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246765    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.247330 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246945    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.247330 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.246966    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.247330 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.247002    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.248206 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.247942    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.249428 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.249401    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.249870 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.249843    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.250692 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.250597    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:37:52.250692 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.250668    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.251359 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.251321    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-config" (OuterVolumeSpecName: "config") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.251605 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.251567    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.251866 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.251814    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.252425 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.252390    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:37:52.253265 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.253207    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-config-out" (OuterVolumeSpecName: "config-out") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:37:52.253265 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.253208    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.253490 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.253347    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-kube-api-access-cfvwc" (OuterVolumeSpecName: "kube-api-access-cfvwc") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "kube-api-access-cfvwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:37:52.253825 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.253786    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.253902 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.253863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.254140 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.254120    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.254856 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.254828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:37:52.263745 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.263715    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-web-config" (OuterVolumeSpecName: "web-config") pod "82ab2a6c-ec16-4254-9bb5-276d53845c61" (UID: "82ab2a6c-ec16-4254-9bb5-276d53845c61"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:37:52.348276 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348242    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-config-out\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348276 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348269    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-web-config\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348276 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348281    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348291    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfvwc\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-kube-api-access-cfvwc\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348300    2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-grpc-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348308    2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-metrics-client-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348317    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348327    2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-kube-rbac-proxy\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348335    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348344    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-db\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348353    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-metrics-client-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348361    2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-config\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348369    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348378    2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348385    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82ab2a6c-ec16-4254-9bb5-276d53845c61-tls-assets\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348393    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348402    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82ab2a6c-ec16-4254-9bb5-276d53845c61-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.348475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.348412    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82ab2a6c-ec16-4254-9bb5-276d53845c61-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:37:52.716303 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716271    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307" exitCode=0
Jul 10 14:37:52.716303 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716295    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c" exitCode=0
Jul 10 14:37:52.716303 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716303    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d" exitCode=0
Jul 10 14:37:52.716303 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716309    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e" exitCode=0
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716314    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500" exitCode=0
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716319    2583 generic.go:356] "Generic (PLEG): container finished" podID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d" exitCode=0
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716348    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716352    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"}
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"}
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"}
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"}
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716532    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"}
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716556    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"}
Jul 10 14:37:52.716816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82ab2a6c-ec16-4254-9bb5-276d53845c61","Type":"ContainerDied","Data":"3ed05fdbd925cbace37ed858e8cbbad2c519d1379a924a6450a9042575da5812"}
Jul 10 14:37:52.717384 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.716541    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.725071 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.725055    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.731668 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.731653    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.741820 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.741798    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.742416 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.742393    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:37:52.748863 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.748846    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:37:52.751007 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.750987    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.757505 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.757489    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.764310 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.764294    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.781318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.781294    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.781564 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.781548    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.781623 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.781570    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"} err="failed to get container status \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist"
Jul 10 14:37:52.781623 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.781588    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.781787 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.781772    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.781824 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.781790    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"} err="failed to get container status \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist"
Jul 10 14:37:52.781824 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.781801    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.782011 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.781998    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.782049 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782012    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"} err="failed to get container status \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist"
Jul 10 14:37:52.782049 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782022    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.782236 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.782210    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.782301 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782243    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"} err="failed to get container status \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist"
Jul 10 14:37:52.782301 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782265    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.782473 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.782458    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.782526 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782475    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"} err="failed to get container status \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist"
Jul 10 14:37:52.782526 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782485    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.782801 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.782788    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.782862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782804    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"} err="failed to get container status \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist"
Jul 10 14:37:52.782862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782816    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.782995 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:37:52.782979    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.783029 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.782998    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"} err="failed to get container status \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist"
Jul 10 14:37:52.783029 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783010    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.783213 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783195    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"} err="failed to get container status \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist"
Jul 10 14:37:52.783259 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783214    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.783384 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783369    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"} err="failed to get container status \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist"
Jul 10 14:37:52.783427 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783384    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.783575 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783560    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"} err="failed to get container status \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist"
Jul 10 14:37:52.783622 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783574    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.783811 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783793    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"} err="failed to get container status \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist"
Jul 10 14:37:52.783864 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783812    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.783999 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783984    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"} err="failed to get container status \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist"
Jul 10 14:37:52.784038 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.783999    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.784202 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784183    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"} err="failed to get container status \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist"
Jul 10 14:37:52.784272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784204    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.784399 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784383    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"} err="failed to get container status \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist"
Jul 10 14:37:52.784445 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784401    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.784610 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784593    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"} err="failed to get container status \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist"
Jul 10 14:37:52.784659 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784610    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.784802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784788    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"} err="failed to get container status \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist"
Jul 10 14:37:52.784850 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784802    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.784978 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784962    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"} err="failed to get container status \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist"
Jul 10 14:37:52.785014 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.784977    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.785147 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785131    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"} err="failed to get container status \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist"
Jul 10 14:37:52.785147 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785146    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.785309 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785292    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"} err="failed to get container status \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist"
Jul 10 14:37:52.785354 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785310    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.785510 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785496    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"} err="failed to get container status \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist"
Jul 10 14:37:52.785510 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785509    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.785732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785712    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"} err="failed to get container status \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist"
Jul 10 14:37:52.785778 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785733    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.785922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785907    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"} err="failed to get container status \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist"
Jul 10 14:37:52.785961 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.785924    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.786188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786140    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"} err="failed to get container status \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist"
Jul 10 14:37:52.786188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786156    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.786446 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786354    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"} err="failed to get container status \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist"
Jul 10 14:37:52.786446 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786375    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.787333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786670    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"} err="failed to get container status \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist"
Jul 10 14:37:52.787333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786691    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.787333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786926    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"} err="failed to get container status \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist"
Jul 10 14:37:52.787333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.786941    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.787333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787250    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"} err="failed to get container status \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist"
Jul 10 14:37:52.787333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787267    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.787670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787484    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"} err="failed to get container status \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist"
Jul 10 14:37:52.787670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787511    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.787769 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787722    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"} err="failed to get container status \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist"
Jul 10 14:37:52.787769 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787740    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.787983 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787963    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"} err="failed to get container status \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist"
Jul 10 14:37:52.788049 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.787983    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.788204 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788186    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"} err="failed to get container status \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist"
Jul 10 14:37:52.788259 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788205    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.788411 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788392    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:37:52.788463 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788409    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"} err="failed to get container status \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist"
Jul 10 14:37:52.788463 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788423    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.788650 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788634    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"} err="failed to get container status \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist"
Jul 10 14:37:52.788650 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788649    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.788789 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788775    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="thanos-sidecar"
Jul 10 14:37:52.788846 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788794    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="thanos-sidecar"
Jul 10 14:37:52.788846 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788807    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy"
Jul 10 14:37:52.788846 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788816    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy"
Jul 10 14:37:52.788846 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788835    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-thanos"
Jul 10 14:37:52.788846 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788838    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"} err="failed to get container status \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788855    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788843    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-thanos"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788898    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="init-config-reloader"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788906    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="init-config-reloader"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788918    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-web"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788927    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-web"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788945    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="config-reloader"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788950    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="config-reloader"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788957    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="prometheus"
Jul 10 14:37:52.789018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.788962    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="prometheus"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789026    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="config-reloader"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789035    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-web"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789042    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789050    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="prometheus"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789059    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="kube-rbac-proxy-thanos"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789068    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" containerName="thanos-sidecar"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789075    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"} err="failed to get container status \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789092    2583 scope.go:117] "RemoveContainer" containerID="18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789285    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307"} err="failed to get container status \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": rpc error: code = NotFound desc = could not find container \"18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307\": container with ID starting with 18cd213181b481bfa1a68f29a9f8ac8b3682fcfcaf85d4ac8533f84c7d123307 not found: ID does not exist"
Jul 10 14:37:52.789394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789310    2583 scope.go:117] "RemoveContainer" containerID="aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"
Jul 10 14:37:52.789765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789497    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c"} err="failed to get container status \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": rpc error: code = NotFound desc = could not find container \"aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c\": container with ID starting with aa85e2fed1713559fd93007ac60bf6326f0f635340cafb888ea116ae66e84f6c not found: ID does not exist"
Jul 10 14:37:52.789765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789541    2583 scope.go:117] "RemoveContainer" containerID="bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"
Jul 10 14:37:52.789765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789712    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d"} err="failed to get container status \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": rpc error: code = NotFound desc = could not find container \"bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d\": container with ID starting with bbc01361c255ca083f4fdb7600e0e41c0fc63be2b728b40f91ca4b155e5daa4d not found: ID does not exist"
Jul 10 14:37:52.789765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789724    2583 scope.go:117] "RemoveContainer" containerID="d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"
Jul 10 14:37:52.789918 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789882    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e"} err="failed to get container status \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": rpc error: code = NotFound desc = could not find container \"d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e\": container with ID starting with d7ff1a5eeb12ab5ff9e1d0a45b972299b763d9ccc07dc8a9f4262dec8b1a150e not found: ID does not exist"
Jul 10 14:37:52.789918 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.789897    2583 scope.go:117] "RemoveContainer" containerID="4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"
Jul 10 14:37:52.790142 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.790125    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500"} err="failed to get container status \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": rpc error: code = NotFound desc = could not find container \"4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500\": container with ID starting with 4d9ed7ed301ee26af02ec9dc6691e24e041b1414a7915b531cbf840c235aa500 not found: ID does not exist"
Jul 10 14:37:52.790198 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.790145    2583 scope.go:117] "RemoveContainer" containerID="4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"
Jul 10 14:37:52.790378 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.790362    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d"} err="failed to get container status \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": rpc error: code = NotFound desc = could not find container \"4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d\": container with ID starting with 4e75335f4c7a9b40849b33835949595f3f751118d286b33763a46b315179779d not found: ID does not exist"
Jul 10 14:37:52.790415 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.790378    2583 scope.go:117] "RemoveContainer" containerID="3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"
Jul 10 14:37:52.790618 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.790600    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84"} err="failed to get container status \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": rpc error: code = NotFound desc = could not find container \"3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84\": container with ID starting with 3627cfde0ea14c04e0aa2a3cac008d4cee343a23b87d40b127c9fe5f85fd5d84 not found: ID does not exist"
Jul 10 14:37:52.792842 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.792829    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.797669 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.797649    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 14:37:52.797772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.797676    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 14:37:52.797772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.797758    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 14:37:52.797884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.797793    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 14:37:52.797884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.797775    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 14:37:52.798084 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798070    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 14:37:52.798296 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798278    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mf8t7\""
Jul 10 14:37:52.798460 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798439    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 14:37:52.798565 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798530    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 14:37:52.798638 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798570    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 14:37:52.798638 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798618    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 14:37:52.798973 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.798956    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 14:37:52.799079 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.799040    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f5jnnl3o8tnje\""
Jul 10 14:37:52.799500 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.799351    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 14:37:52.799937 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.799920    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 14:37:52.801851 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.801834    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 14:37:52.807270 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.807254    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 14:37:52.815230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.815174    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:37:52.844444 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.844420    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ab2a6c-ec16-4254-9bb5-276d53845c61" path="/var/lib/kubelet/pods/82ab2a6c-ec16-4254-9bb5-276d53845c61/volumes"
Jul 10 14:37:52.852715 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852773 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852773 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852773 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852757    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852876 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852876 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4cf1b68-d3e6-42ca-9067-33bdbd191917-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852876 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852861    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852963 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852963 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-web-config\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.852963 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852961    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.852984    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4cf1b68-d3e6-42ca-9067-33bdbd191917-config-out\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853160 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853160 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853098    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853160 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853160 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frwb8\" (UniqueName: \"kubernetes.io/projected/e4cf1b68-d3e6-42ca-9067-33bdbd191917-kube-api-access-frwb8\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853160 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-config\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.853295 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.853168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-web-config\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953765 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4cf1b68-d3e6-42ca-9067-33bdbd191917-config-out\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.953926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953896    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.953979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frwb8\" (UniqueName: \"kubernetes.io/projected/e4cf1b68-d3e6-42ca-9067-33bdbd191917-kube-api-access-frwb8\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-config\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954060    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954622 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954175    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954622 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4cf1b68-d3e6-42ca-9067-33bdbd191917-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954622 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954622 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954284    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954866 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954839    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.954932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.954918    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.955396 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.955369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.956739 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.956674    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4cf1b68-d3e6-42ca-9067-33bdbd191917-config-out\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.957442 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.956885    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.957442 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.957146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-web-config\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.957442 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.957153    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.957562    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.957650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.957701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.957830    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.958120    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.958240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.958431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.958289    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf1b68-d3e6-42ca-9067-33bdbd191917-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.959406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.959379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.959494 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.959445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.959600 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.959581    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-config\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.960177 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.960155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4cf1b68-d3e6-42ca-9067-33bdbd191917-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.960277 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.960262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e4cf1b68-d3e6-42ca-9067-33bdbd191917-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:52.967331 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:52.967281    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frwb8\" (UniqueName: \"kubernetes.io/projected/e4cf1b68-d3e6-42ca-9067-33bdbd191917-kube-api-access-frwb8\") pod \"prometheus-k8s-0\" (UID: \"e4cf1b68-d3e6-42ca-9067-33bdbd191917\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:53.102704 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:53.102674    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:37:53.240022 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:53.239945    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:37:53.243035 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:37:53.243009    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cf1b68_d3e6_42ca_9067_33bdbd191917.slice/crio-f1c512c8f0eb44b682ad5b2451319e64ab4f01b03015c80796bab0db3a870d36 WatchSource:0}: Error finding container f1c512c8f0eb44b682ad5b2451319e64ab4f01b03015c80796bab0db3a870d36: Status 404 returned error can't find the container with id f1c512c8f0eb44b682ad5b2451319e64ab4f01b03015c80796bab0db3a870d36
Jul 10 14:37:53.722200 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:53.722170    2583 generic.go:356] "Generic (PLEG): container finished" podID="e4cf1b68-d3e6-42ca-9067-33bdbd191917" containerID="d2fd8c033b0253188f0cd0dc180b1301b0e77b321f908e2f028d60e59b4b8f90" exitCode=0
Jul 10 14:37:53.722666 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:53.722259    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerDied","Data":"d2fd8c033b0253188f0cd0dc180b1301b0e77b321f908e2f028d60e59b4b8f90"}
Jul 10 14:37:53.722666 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:53.722364    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"f1c512c8f0eb44b682ad5b2451319e64ab4f01b03015c80796bab0db3a870d36"}
Jul 10 14:37:54.730219 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.730182    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"e2c3776e7411b3edf7c3127d15434275127792dd454ecaaff3301656c6b96693"}
Jul 10 14:37:54.730597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.730270    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"7f5b234b06b7997611ed5b2c10ff92a349ed2ce91e1ee5f4116b6c596d14be37"}
Jul 10 14:37:54.730597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.730310    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"917b6e4749d1304f000809217c74456e06ac2feecf2357cfbe7e66a85d684a7d"}
Jul 10 14:37:54.730597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.730353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"8c909f2a58e011a18ee303bcf542e7fadf871ccb9a044c1c0c89cb8de87d529f"}
Jul 10 14:37:54.730597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.730388    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"ab700dadf229f13e18528c39f7c1abdd9755e33d9bfa39bec99020670b1641ec"}
Jul 10 14:37:54.730597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.730424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4cf1b68-d3e6-42ca-9067-33bdbd191917","Type":"ContainerStarted","Data":"5263bb467ee274c6b50f341719ea06c3da1fb03f2a75fb3c1ac4f24d540037ed"}
Jul 10 14:37:54.764042 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:54.763988    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.746610832 podStartE2EDuration="2.763969517s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.358685ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:37:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:37:54.76197464 +0000 UTC m=+172.468296147" watchObservedRunningTime="2026-07-10 14:37:54.763969517 +0000 UTC m=+172.470290988"
Jul 10 14:37:55.280159 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:55.280131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:55.382445 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:55.382418    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:58.103170 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:37:58.103139    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:38:04.550405 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:04.550369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:04.613171 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:04.613138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:10.327335 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:10.327299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:10.427422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:10.427388    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:15.160989 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:15.160954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:15.220451 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:15.220422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:15.280451 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:15.280426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:21.693934 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:21.693903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:25.611615 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:25.611586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:25.712747 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:25.712713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:53.103310 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:53.103276    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:38:53.117688 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:53.117665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:38:53.957732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:53.957704    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:38:58.110169 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:58.110131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:58.212121 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:38:58.212084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:24.763351 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:24.763271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:25.803141 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:25.803097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:28.369397 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:28.369363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:28.439875 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:28.439842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:33.027834 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:33.027800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:33.106860 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:33.106825    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:44.396748 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:44.396711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:44.574895 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:44.574866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:44.976398 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:44.976365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:45.172069 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:45.172041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:45.855078 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:45.855049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:45.959761 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:45.959731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:46.562863 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:46.562831    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:46.970767 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:46.970737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:54.523352 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:39:54.523318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:00.302396 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:00.302358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:00.399811 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:00.399779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:01.987479 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:01.987443    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:02.656343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:02.656316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:40:02.658606 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:02.658545    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:40:02.661853 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:02.661827    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 14:40:04.017973 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.017943    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"]
Jul 10 14:40:04.020165 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.020151    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.023351 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.023333    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 14:40:04.024246 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.024227    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 14:40:04.024349 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.024294    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rh2xh\""
Jul 10 14:40:04.032357 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.032336    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"]
Jul 10 14:40:04.145883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.145853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c319c34-989c-4ea8-b7ce-216cc73f799e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k62hf\" (UID: \"5c319c34-989c-4ea8-b7ce-216cc73f799e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.146012 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.145893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbndz\" (UniqueName: \"kubernetes.io/projected/5c319c34-989c-4ea8-b7ce-216cc73f799e-kube-api-access-xbndz\") pod \"openshift-lws-operator-fd8ccff4c-k62hf\" (UID: \"5c319c34-989c-4ea8-b7ce-216cc73f799e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.246450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.246420    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c319c34-989c-4ea8-b7ce-216cc73f799e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k62hf\" (UID: \"5c319c34-989c-4ea8-b7ce-216cc73f799e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.246599 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.246462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbndz\" (UniqueName: \"kubernetes.io/projected/5c319c34-989c-4ea8-b7ce-216cc73f799e-kube-api-access-xbndz\") pod \"openshift-lws-operator-fd8ccff4c-k62hf\" (UID: \"5c319c34-989c-4ea8-b7ce-216cc73f799e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.246849 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.246829    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c319c34-989c-4ea8-b7ce-216cc73f799e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k62hf\" (UID: \"5c319c34-989c-4ea8-b7ce-216cc73f799e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.271311 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.271256    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbndz\" (UniqueName: \"kubernetes.io/projected/5c319c34-989c-4ea8-b7ce-216cc73f799e-kube-api-access-xbndz\") pod \"openshift-lws-operator-fd8ccff4c-k62hf\" (UID: \"5c319c34-989c-4ea8-b7ce-216cc73f799e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.329115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.329095    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"
Jul 10 14:40:04.460016 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.459994    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf"]
Jul 10 14:40:04.461952 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:04.461926    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c319c34_989c_4ea8_b7ce_216cc73f799e.slice/crio-5f2acc1d4fb3c894ada8141305caacf8757731b0c750ad26d503eadd0eac3b04 WatchSource:0}: Error finding container 5f2acc1d4fb3c894ada8141305caacf8757731b0c750ad26d503eadd0eac3b04: Status 404 returned error can't find the container with id 5f2acc1d4fb3c894ada8141305caacf8757731b0c750ad26d503eadd0eac3b04
Jul 10 14:40:04.463352 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.463338    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:40:04.519445 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:04.519419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:05.203078 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:05.203044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf" event={"ID":"5c319c34-989c-4ea8-b7ce-216cc73f799e","Type":"ContainerStarted","Data":"5f2acc1d4fb3c894ada8141305caacf8757731b0c750ad26d503eadd0eac3b04"}
Jul 10 14:40:07.213169 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:07.213122    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf" event={"ID":"5c319c34-989c-4ea8-b7ce-216cc73f799e","Type":"ContainerStarted","Data":"10d69389c0276317ee04ff0025968c0b67df0f87496bad783e54113b28ce9df8"}
Jul 10 14:40:07.249732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:07.249685    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k62hf" podStartSLOduration=1.6185444260000001 podStartE2EDuration="4.249669811s" totalImagesPullingTime="2.631125385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:07.249415652 +0000 UTC m=+304.955737123" watchObservedRunningTime="2026-07-10 14:40:07.249669811 +0000 UTC m=+304.955991281"
Jul 10 14:40:08.059357 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:08.059325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:08.158826 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:08.158790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:10.406327 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:10.406290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.792627 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:22.792595    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.953854 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:22.953810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:23.394333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.394286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:23.620326 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.620293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:23.730376 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.730350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:23.841191 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.841161    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"]
Jul 10 14:40:23.844778 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.844762    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:23.848621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.848600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 14:40:23.848745 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.848719    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 14:40:23.848817 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.848688    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 14:40:23.849131 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.849115    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rpbct\""
Jul 10 14:40:23.856452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.856435    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 14:40:23.875343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.875317    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"]
Jul 10 14:40:23.901351 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.901325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x7ss\" (UniqueName: \"kubernetes.io/projected/e9febbf1-8597-481a-84f2-c344b636cd41-kube-api-access-2x7ss\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:23.901452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.901382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9febbf1-8597-481a-84f2-c344b636cd41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:23.901452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.901408    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9febbf1-8597-481a-84f2-c344b636cd41-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:23.901452 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.901440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9febbf1-8597-481a-84f2-c344b636cd41-tmp\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:23.901619 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:23.901462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9febbf1-8597-481a-84f2-c344b636cd41-manifests\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.002302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.002206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9febbf1-8597-481a-84f2-c344b636cd41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.002302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.002245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9febbf1-8597-481a-84f2-c344b636cd41-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.002302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.002289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9febbf1-8597-481a-84f2-c344b636cd41-tmp\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.002530 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.002310    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9febbf1-8597-481a-84f2-c344b636cd41-manifests\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.002530 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.002348    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x7ss\" (UniqueName: \"kubernetes.io/projected/e9febbf1-8597-481a-84f2-c344b636cd41-kube-api-access-2x7ss\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.002907 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.002873    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9febbf1-8597-481a-84f2-c344b636cd41-manifests\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.004694 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.004670    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9febbf1-8597-481a-84f2-c344b636cd41-tmp\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.004798 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.004783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9febbf1-8597-481a-84f2-c344b636cd41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.004906 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.004883    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9febbf1-8597-481a-84f2-c344b636cd41-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.031308 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.031284    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x7ss\" (UniqueName: \"kubernetes.io/projected/e9febbf1-8597-481a-84f2-c344b636cd41-kube-api-access-2x7ss\") pod \"opendatahub-operator-controller-manager-85d85459f4-8xmv4\" (UID: \"e9febbf1-8597-481a-84f2-c344b636cd41\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.155735 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.155698    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:24.240208 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.240160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:24.310321 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.310299    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"]
Jul 10 14:40:24.312778 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:24.312747    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9febbf1_8597_481a_84f2_c344b636cd41.slice/crio-9acb19348a6ca1e74aabd8267cccfd0a4d6c717ca49c6c1812f7c9a62f84b064 WatchSource:0}: Error finding container 9acb19348a6ca1e74aabd8267cccfd0a4d6c717ca49c6c1812f7c9a62f84b064: Status 404 returned error can't find the container with id 9acb19348a6ca1e74aabd8267cccfd0a4d6c717ca49c6c1812f7c9a62f84b064
Jul 10 14:40:24.464903 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.464866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:24.592409 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.592321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:24.732656 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.732624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:24.897836 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:24.897733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:25.018258 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:25.018171    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:25.283566 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:25.283530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4" event={"ID":"e9febbf1-8597-481a-84f2-c344b636cd41","Type":"ContainerStarted","Data":"9acb19348a6ca1e74aabd8267cccfd0a4d6c717ca49c6c1812f7c9a62f84b064"}
Jul 10 14:40:28.297137 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:28.297102    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4" event={"ID":"e9febbf1-8597-481a-84f2-c344b636cd41","Type":"ContainerStarted","Data":"069f798761b222ca13e32512362ff1f7af4d99273599993fa1d8271c4ac71e83"}
Jul 10 14:40:29.302868 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:29.302837    2583 generic.go:356] "Generic (PLEG): container finished" podID="e9febbf1-8597-481a-84f2-c344b636cd41" containerID="069f798761b222ca13e32512362ff1f7af4d99273599993fa1d8271c4ac71e83" exitCode=0
Jul 10 14:40:29.303257 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:29.302898    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4" event={"ID":"e9febbf1-8597-481a-84f2-c344b636cd41","Type":"ContainerDied","Data":"069f798761b222ca13e32512362ff1f7af4d99273599993fa1d8271c4ac71e83"}
Jul 10 14:40:30.309376 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:30.309339    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4" event={"ID":"e9febbf1-8597-481a-84f2-c344b636cd41","Type":"ContainerStarted","Data":"0267c237296343211f631b8711b07aa9f8d2a0133e2eb0d1f08fce6feb1ad596"}
Jul 10 14:40:30.309785 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:30.309478    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:30.334377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:30.334331    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4" podStartSLOduration=2.990470943 podStartE2EDuration="7.334316376s" totalImagesPullingTime="3.338139139s" totalInitContainerRuntime="1.005706294s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:30.333752341 +0000 UTC m=+328.040073809" watchObservedRunningTime="2026-07-10 14:40:30.334316376 +0000 UTC m=+328.040637844"
Jul 10 14:40:35.015649 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:35.015617    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:35.115881 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:35.115853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:41.316331 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:41.316303    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-8xmv4"
Jul 10 14:40:43.082128 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.082046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:43.199270 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.199234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:43.754051 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.754020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:43.868618 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.868586    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"]
Jul 10 14:40:43.880731 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.880701    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:43.885500 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.885477    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 14:40:43.886365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.886344    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 14:40:43.886509 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.886493    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 14:40:43.887554 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.887510    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 14:40:43.887646 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.887569    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ghdzt\""
Jul 10 14:40:43.894798 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.894781    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"]
Jul 10 14:40:43.957130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.957097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hlfv\" (UniqueName: \"kubernetes.io/projected/0186c81d-ae51-4005-81fe-ca87633d1494-kube-api-access-8hlfv\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:43.957259 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.957178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0186c81d-ae51-4005-81fe-ca87633d1494-tmp\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:43.957259 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:43.957217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0186c81d-ae51-4005-81fe-ca87633d1494-tls-certs\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.058067 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.058005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0186c81d-ae51-4005-81fe-ca87633d1494-tmp\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.058067 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.058041    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0186c81d-ae51-4005-81fe-ca87633d1494-tls-certs\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.058262 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.058080    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hlfv\" (UniqueName: \"kubernetes.io/projected/0186c81d-ae51-4005-81fe-ca87633d1494-kube-api-access-8hlfv\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.058262 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:44.058186    2583 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 10 14:40:44.058378 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:44.058279    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0186c81d-ae51-4005-81fe-ca87633d1494-tls-certs podName:0186c81d-ae51-4005-81fe-ca87633d1494 nodeName:}" failed. No retries permitted until 2026-07-10 14:40:44.558257204 +0000 UTC m=+342.264578652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0186c81d-ae51-4005-81fe-ca87633d1494-tls-certs") pod "kube-auth-proxy-74544d9668-p2dhc" (UID: "0186c81d-ae51-4005-81fe-ca87633d1494") : secret "kube-auth-proxy-tls" not found
Jul 10 14:40:44.060030 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.060012    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0186c81d-ae51-4005-81fe-ca87633d1494-tmp\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.077877 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.077855    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hlfv\" (UniqueName: \"kubernetes.io/projected/0186c81d-ae51-4005-81fe-ca87633d1494-kube-api-access-8hlfv\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.101907 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.101881    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zbx64"]
Jul 10 14:40:44.104326 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.104307    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.108463 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.108443    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 10 14:40:44.108670 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.108658    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9wmlf\""
Jul 10 14:40:44.132745 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.132722    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zbx64"]
Jul 10 14:40:44.158786 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.158761    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29nmk\" (UniqueName: \"kubernetes.io/projected/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-kube-api-access-29nmk\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.158868 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.158839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-cert\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.210138 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.210113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:44.259424 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.259399    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-cert\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.259572 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:44.259550    2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 10 14:40:44.259637 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.259569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29nmk\" (UniqueName: \"kubernetes.io/projected/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-kube-api-access-29nmk\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.259637 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:44.259610    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-cert podName:51a6fdb3-a4da-4ac3-b8b3-5f234ade7048 nodeName:}" failed. No retries permitted until 2026-07-10 14:40:44.759591069 +0000 UTC m=+342.465912535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-cert") pod "odh-model-controller-7998c67dc4-zbx64" (UID: "51a6fdb3-a4da-4ac3-b8b3-5f234ade7048") : secret "odh-model-controller-webhook-cert" not found
Jul 10 14:40:44.275927 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.275902    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29nmk\" (UniqueName: \"kubernetes.io/projected/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-kube-api-access-29nmk\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.346064 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.345998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:44.561641 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.561609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0186c81d-ae51-4005-81fe-ca87633d1494-tls-certs\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.564416 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.564391    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0186c81d-ae51-4005-81fe-ca87633d1494-tls-certs\") pod \"kube-auth-proxy-74544d9668-p2dhc\" (UID: \"0186c81d-ae51-4005-81fe-ca87633d1494\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.601555 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.601472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:44.699455 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.699431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:44.763414 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.763385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-cert\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.765665 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.765645    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a6fdb3-a4da-4ac3-b8b3-5f234ade7048-cert\") pod \"odh-model-controller-7998c67dc4-zbx64\" (UID: \"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048\") " pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:44.790507 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.790483    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"
Jul 10 14:40:44.936001 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:44.935977    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74544d9668-p2dhc"]
Jul 10 14:40:44.937856 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:44.937832    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0186c81d_ae51_4005_81fe_ca87633d1494.slice/crio-19de83eb29292107596868883e486e1df0e41a554098454caa30f5fa39ba5da7 WatchSource:0}: Error finding container 19de83eb29292107596868883e486e1df0e41a554098454caa30f5fa39ba5da7: Status 404 returned error can't find the container with id 19de83eb29292107596868883e486e1df0e41a554098454caa30f5fa39ba5da7
Jul 10 14:40:45.014142 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.014119    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:45.177862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.177828    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zbx64"]
Jul 10 14:40:45.181670 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:45.181629    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a6fdb3_a4da_4ac3_b8b3_5f234ade7048.slice/crio-f5d5dbf5eb10d7f3f85e43ebaf52d9094028d3cf98f74dd024978fd4f86575b9 WatchSource:0}: Error finding container f5d5dbf5eb10d7f3f85e43ebaf52d9094028d3cf98f74dd024978fd4f86575b9: Status 404 returned error can't find the container with id f5d5dbf5eb10d7f3f85e43ebaf52d9094028d3cf98f74dd024978fd4f86575b9
Jul 10 14:40:45.255425 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.255397    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rqg5c"]
Jul 10 14:40:45.258099 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.258084    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.262696 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.262678    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 14:40:45.279841 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.279786    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rqg5c"]
Jul 10 14:40:45.280693 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.280670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:45.370280 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.370247    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrjj\" (UniqueName: \"kubernetes.io/projected/b85bd97c-19c6-45ea-9fa7-3505181a3293-kube-api-access-8jrjj\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.370460 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.370335    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85bd97c-19c6-45ea-9fa7-3505181a3293-tls-certs\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.373068 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.373040    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc" event={"ID":"0186c81d-ae51-4005-81fe-ca87633d1494","Type":"ContainerStarted","Data":"19de83eb29292107596868883e486e1df0e41a554098454caa30f5fa39ba5da7"}
Jul 10 14:40:45.374073 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.374054    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64" event={"ID":"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048","Type":"ContainerStarted","Data":"f5d5dbf5eb10d7f3f85e43ebaf52d9094028d3cf98f74dd024978fd4f86575b9"}
Jul 10 14:40:45.471790 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.471708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrjj\" (UniqueName: \"kubernetes.io/projected/b85bd97c-19c6-45ea-9fa7-3505181a3293-kube-api-access-8jrjj\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.471790 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.471790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85bd97c-19c6-45ea-9fa7-3505181a3293-tls-certs\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.471974 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:45.471904    2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 10 14:40:45.471974 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:45.471968    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85bd97c-19c6-45ea-9fa7-3505181a3293-tls-certs podName:b85bd97c-19c6-45ea-9fa7-3505181a3293 nodeName:}" failed. No retries permitted until 2026-07-10 14:40:45.971947816 +0000 UTC m=+343.678269279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b85bd97c-19c6-45ea-9fa7-3505181a3293-tls-certs") pod "model-serving-api-54d89bd5fc-rqg5c" (UID: "b85bd97c-19c6-45ea-9fa7-3505181a3293") : secret "model-serving-api-tls" not found
Jul 10 14:40:45.484559 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.484512    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrjj\" (UniqueName: \"kubernetes.io/projected/b85bd97c-19c6-45ea-9fa7-3505181a3293-kube-api-access-8jrjj\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.716808 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.716780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:45.777628 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.777550    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"]
Jul 10 14:40:45.780256 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.780233    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.783127 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.783094    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 14:40:45.783530 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.783503    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bmh45\""
Jul 10 14:40:45.783660 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.783574    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 14:40:45.783660 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.783648    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 14:40:45.783812 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.783793    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 14:40:45.798766 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.798714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"]
Jul 10 14:40:45.804399 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.804373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:45.875604 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875576    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8pzc\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-kube-api-access-n8pzc\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.875758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.875758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fccd4159-1f73-4a33-bc34-a81b84f14d15-local-certs\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.875758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.875758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875734    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.875972 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875778    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-cacerts\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.875972 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.875842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-token\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.930132 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.930096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:45.976685 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.976839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fccd4159-1f73-4a33-bc34-a81b84f14d15-local-certs\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.976839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.976839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.976839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-cacerts\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.977091 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976858    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-token\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.977091 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85bd97c-19c6-45ea-9fa7-3505181a3293-tls-certs\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.977091 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.976925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8pzc\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-kube-api-access-n8pzc\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.978475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.978411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.982055 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.982014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.982499 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.982463    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-cacerts\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.982609 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.982573    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85bd97c-19c6-45ea-9fa7-3505181a3293-tls-certs\") pod \"model-serving-api-54d89bd5fc-rqg5c\" (UID: \"b85bd97c-19c6-45ea-9fa7-3505181a3293\") " pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:45.982774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.982750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:45.982983 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:45.982946    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fccd4159-1f73-4a33-bc34-a81b84f14d15-local-certs\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:46.001063 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.001041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-token\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:46.002773 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.002751    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8pzc\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-kube-api-access-n8pzc\") pod \"istiod-openshift-gateway-7574459656-6p2l5\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:46.093428 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.093366    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:46.167694 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.167660    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:46.279588 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.279540    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"]
Jul 10 14:40:46.287156 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:46.285151    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfccd4159_1f73_4a33_bc34_a81b84f14d15.slice/crio-893c4bab515c308de2fe375bfd4d10fc858156b5d71e4fa277e8be77f2a6e4ff WatchSource:0}: Error finding container 893c4bab515c308de2fe375bfd4d10fc858156b5d71e4fa277e8be77f2a6e4ff: Status 404 returned error can't find the container with id 893c4bab515c308de2fe375bfd4d10fc858156b5d71e4fa277e8be77f2a6e4ff
Jul 10 14:40:46.366012 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:46.365977    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb85bd97c_19c6_45ea_9fa7_3505181a3293.slice/crio-46473e05616071817bc11f016fc22239853edc1b49a83101c3bc1c229c6f5bd7 WatchSource:0}: Error finding container 46473e05616071817bc11f016fc22239853edc1b49a83101c3bc1c229c6f5bd7: Status 404 returned error can't find the container with id 46473e05616071817bc11f016fc22239853edc1b49a83101c3bc1c229c6f5bd7
Jul 10 14:40:46.368190 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.368166    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rqg5c"]
Jul 10 14:40:46.380289 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.380254    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" event={"ID":"fccd4159-1f73-4a33-bc34-a81b84f14d15","Type":"ContainerStarted","Data":"893c4bab515c308de2fe375bfd4d10fc858156b5d71e4fa277e8be77f2a6e4ff"}
Jul 10 14:40:46.381649 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:46.381606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c" event={"ID":"b85bd97c-19c6-45ea-9fa7-3505181a3293","Type":"ContainerStarted","Data":"46473e05616071817bc11f016fc22239853edc1b49a83101c3bc1c229c6f5bd7"}
Jul 10 14:40:48.199015 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:48.198953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:50.202676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.202638    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:40:50.203041 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.202713    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:40:50.404246 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.404214    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c" event={"ID":"b85bd97c-19c6-45ea-9fa7-3505181a3293","Type":"ContainerStarted","Data":"7c17820c4617c28a905ef7b42d42a4ba5b3b2b2157cf6d0d7f958eb3093cfa16"}
Jul 10 14:40:50.404421 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.404306    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:40:50.405820 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.405794    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc" event={"ID":"0186c81d-ae51-4005-81fe-ca87633d1494","Type":"ContainerStarted","Data":"ab2a56cd82ec644b81f51e91700bb90075e2d83d55775292328db93b940b1a8b"}
Jul 10 14:40:50.407232 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.407212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64" event={"ID":"51a6fdb3-a4da-4ac3-b8b3-5f234ade7048","Type":"ContainerStarted","Data":"fdb9641afc26dff004c94ace229b1ef505c9f6193f30f3da59ba2368e1d6631c"}
Jul 10 14:40:50.407353 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.407339    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:40:50.408725 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.408708    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" event={"ID":"fccd4159-1f73-4a33-bc34-a81b84f14d15","Type":"ContainerStarted","Data":"1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645"}
Jul 10 14:40:50.408821 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.408810    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:50.477225 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.477136    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c" podStartSLOduration=1.650211041 podStartE2EDuration="5.477120076s" totalImagesPullingTime="3.826909035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:50.433204954 +0000 UTC m=+348.139526422" watchObservedRunningTime="2026-07-10 14:40:50.477120076 +0000 UTC m=+348.183441549"
Jul 10 14:40:50.477561 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.477501    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" podStartSLOduration=1.566400298 podStartE2EDuration="5.477492043s" totalImagesPullingTime="3.911091745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:50.476021755 +0000 UTC m=+348.182343225" watchObservedRunningTime="2026-07-10 14:40:50.477492043 +0000 UTC m=+348.183813510"
Jul 10 14:40:50.531240 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.531191    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74544d9668-p2dhc" podStartSLOduration=2.277243899 podStartE2EDuration="7.531172973s" totalImagesPullingTime="5.253929074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:50.529287243 +0000 UTC m=+348.235608709" watchObservedRunningTime="2026-07-10 14:40:50.531172973 +0000 UTC m=+348.237494444"
Jul 10 14:40:50.592312 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.592261    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64" podStartSLOduration=1.584321453 podStartE2EDuration="6.592242918s" totalImagesPullingTime="5.007921465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:50.589863491 +0000 UTC m=+348.296184959" watchObservedRunningTime="2026-07-10 14:40:50.592242918 +0000 UTC m=+348.298564385"
Jul 10 14:40:50.920010 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:50.919977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:51.083426 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:51.083392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:51.415265 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:51.415239    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-6p2l5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 14:40:51.415648 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:51.415285    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" podUID="fccd4159-1f73-4a33-bc34-a81b84f14d15" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 14:40:52.323758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:52.323715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:52.689146 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:52.689115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:52.835535 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:52.835485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:54.414000 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:54.413925    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:56.651638 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.651608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:56.832802 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.832773    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"]
Jul 10 14:40:56.835839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.835821    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:56.852852 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.852832    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 10 14:40:56.853324 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.853309    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zhp4n\""
Jul 10 14:40:56.881005 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.880984    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"]
Jul 10 14:40:56.883370 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.883348    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"]
Jul 10 14:40:56.887074 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.887054    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:56.890883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.890863    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 10 14:40:56.891205 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.891191    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ttk4v\""
Jul 10 14:40:56.907427 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.907381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:56.963113 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.963088    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"]
Jul 10 14:40:56.974229 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.974197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lq8l\" (UniqueName: \"kubernetes.io/projected/f23f7574-ffe1-474e-b36c-617f367240b0-kube-api-access-9lq8l\") pod \"llmisvc-controller-manager-777c6bd7d5-dl6t6\" (UID: \"f23f7574-ffe1-474e-b36c-617f367240b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:56.974338 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.974239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be021526-cbe6-4f16-a1ac-50d9c38b12ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-mq8jn\" (UID: \"be021526-cbe6-4f16-a1ac-50d9c38b12ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:56.974338 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.974279    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg4x6\" (UniqueName: \"kubernetes.io/projected/be021526-cbe6-4f16-a1ac-50d9c38b12ac-kube-api-access-jg4x6\") pod \"kserve-controller-manager-d6f4b65b8-mq8jn\" (UID: \"be021526-cbe6-4f16-a1ac-50d9c38b12ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:56.974414 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:56.974335    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f23f7574-ffe1-474e-b36c-617f367240b0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dl6t6\" (UID: \"f23f7574-ffe1-474e-b36c-617f367240b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:57.075689 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.075668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f23f7574-ffe1-474e-b36c-617f367240b0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dl6t6\" (UID: \"f23f7574-ffe1-474e-b36c-617f367240b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:57.075812 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.075723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lq8l\" (UniqueName: \"kubernetes.io/projected/f23f7574-ffe1-474e-b36c-617f367240b0-kube-api-access-9lq8l\") pod \"llmisvc-controller-manager-777c6bd7d5-dl6t6\" (UID: \"f23f7574-ffe1-474e-b36c-617f367240b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:57.075812 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.075748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be021526-cbe6-4f16-a1ac-50d9c38b12ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-mq8jn\" (UID: \"be021526-cbe6-4f16-a1ac-50d9c38b12ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:57.075812 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.075765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg4x6\" (UniqueName: \"kubernetes.io/projected/be021526-cbe6-4f16-a1ac-50d9c38b12ac-kube-api-access-jg4x6\") pod \"kserve-controller-manager-d6f4b65b8-mq8jn\" (UID: \"be021526-cbe6-4f16-a1ac-50d9c38b12ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:57.078010 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.077983    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be021526-cbe6-4f16-a1ac-50d9c38b12ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-mq8jn\" (UID: \"be021526-cbe6-4f16-a1ac-50d9c38b12ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:57.078098 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.078040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f23f7574-ffe1-474e-b36c-617f367240b0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dl6t6\" (UID: \"f23f7574-ffe1-474e-b36c-617f367240b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:57.145966 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.145937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg4x6\" (UniqueName: \"kubernetes.io/projected/be021526-cbe6-4f16-a1ac-50d9c38b12ac-kube-api-access-jg4x6\") pod \"kserve-controller-manager-d6f4b65b8-mq8jn\" (UID: \"be021526-cbe6-4f16-a1ac-50d9c38b12ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:57.149235 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.149213    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lq8l\" (UniqueName: \"kubernetes.io/projected/f23f7574-ffe1-474e-b36c-617f367240b0-kube-api-access-9lq8l\") pod \"llmisvc-controller-manager-777c6bd7d5-dl6t6\" (UID: \"f23f7574-ffe1-474e-b36c-617f367240b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:57.198282 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.198241    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:40:57.252275 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.252242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:57.360256 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.360229    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"]
Jul 10 14:40:57.362826 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:57.362794    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe021526_cbe6_4f16_a1ac_50d9c38b12ac.slice/crio-a7b8097d9e7c7f52aaa7f37c6dbf88a4e3e7c3bb37420472c6e3e63d1587a712 WatchSource:0}: Error finding container a7b8097d9e7c7f52aaa7f37c6dbf88a4e3e7c3bb37420472c6e3e63d1587a712: Status 404 returned error can't find the container with id a7b8097d9e7c7f52aaa7f37c6dbf88a4e3e7c3bb37420472c6e3e63d1587a712
Jul 10 14:40:57.445631 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.445599    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:40:57.446869 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.446842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn" event={"ID":"be021526-cbe6-4f16-a1ac-50d9c38b12ac","Type":"ContainerStarted","Data":"a7b8097d9e7c7f52aaa7f37c6dbf88a4e3e7c3bb37420472c6e3e63d1587a712"}
Jul 10 14:40:57.648602 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.648570    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"]
Jul 10 14:40:57.691989 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:40:57.691966    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf23f7574_ffe1_474e_b36c_617f367240b0.slice/crio-2b5582ce42be7d03f9be3ccbf1e93af4f75afc28b024661b8d68ec5d6c7f772b WatchSource:0}: Error finding container 2b5582ce42be7d03f9be3ccbf1e93af4f75afc28b024661b8d68ec5d6c7f772b: Status 404 returned error can't find the container with id 2b5582ce42be7d03f9be3ccbf1e93af4f75afc28b024661b8d68ec5d6c7f772b
Jul 10 14:40:57.885374 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.885302    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"]
Jul 10 14:40:57.885535 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:57.885499    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" podUID="fccd4159-1f73-4a33-bc34-a81b84f14d15" containerName="discovery" containerID="cri-o://1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645" gracePeriod=30
Jul 10 14:40:58.129973 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.129947    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:58.286437 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286406    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-cacerts\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.286621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286462    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fccd4159-1f73-4a33-bc34-a81b84f14d15-local-certs\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.286621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286490    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-kubeconfig\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.286621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286529    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-token\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.286621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286563    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-ca-configmap\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.286621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286597    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8pzc\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-kube-api-access-n8pzc\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.286893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.286669    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-dns-cert\") pod \"fccd4159-1f73-4a33-bc34-a81b84f14d15\" (UID: \"fccd4159-1f73-4a33-bc34-a81b84f14d15\") "
Jul 10 14:40:58.287327 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.287277    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:40:58.289729 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.289695    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-cacerts" (OuterVolumeSpecName: "cacerts") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:40:58.290451 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.290420    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-kube-api-access-n8pzc" (OuterVolumeSpecName: "kube-api-access-n8pzc") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "kube-api-access-n8pzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:40:58.291995 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.291966    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fccd4159-1f73-4a33-bc34-a81b84f14d15-local-certs" (OuterVolumeSpecName: "local-certs") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:40:58.292550 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.292453    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:40:58.292652 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.292605    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:40:58.292652 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.292620    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-token" (OuterVolumeSpecName: "istio-token") pod "fccd4159-1f73-4a33-bc34-a81b84f14d15" (UID: "fccd4159-1f73-4a33-bc34-a81b84f14d15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:40:58.387734 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387703    2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fccd4159-1f73-4a33-bc34-a81b84f14d15-local-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.387734 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387733    2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-kubeconfig\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.387734 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387742    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-token\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.387932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387751    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-ca-configmap\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.387932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387761    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8pzc\" (UniqueName: \"kubernetes.io/projected/fccd4159-1f73-4a33-bc34-a81b84f14d15-kube-api-access-n8pzc\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.387932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387769    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-istio-csr-dns-cert\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.387932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.387779    2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fccd4159-1f73-4a33-bc34-a81b84f14d15-cacerts\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.402277 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.402226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:58.452730 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.452698    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6" event={"ID":"f23f7574-ffe1-474e-b36c-617f367240b0","Type":"ContainerStarted","Data":"2b5582ce42be7d03f9be3ccbf1e93af4f75afc28b024661b8d68ec5d6c7f772b"}
Jul 10 14:40:58.454229 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.454190    2583 generic.go:356] "Generic (PLEG): container finished" podID="fccd4159-1f73-4a33-bc34-a81b84f14d15" containerID="1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645" exitCode=0
Jul 10 14:40:58.454362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.454243    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"
Jul 10 14:40:58.454362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.454272    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" event={"ID":"fccd4159-1f73-4a33-bc34-a81b84f14d15","Type":"ContainerDied","Data":"1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645"}
Jul 10 14:40:58.454362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.454319    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5" event={"ID":"fccd4159-1f73-4a33-bc34-a81b84f14d15","Type":"ContainerDied","Data":"893c4bab515c308de2fe375bfd4d10fc858156b5d71e4fa277e8be77f2a6e4ff"}
Jul 10 14:40:58.454362 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.454350    2583 scope.go:117] "RemoveContainer" containerID="1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645"
Jul 10 14:40:58.494721 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.494688    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"]
Jul 10 14:40:58.499495 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.499466    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6p2l5"]
Jul 10 14:40:58.528076 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.528056    2583 scope.go:117] "RemoveContainer" containerID="1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645"
Jul 10 14:40:58.528416 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:40:58.528392    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645\": container with ID starting with 1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645 not found: ID does not exist" containerID="1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645"
Jul 10 14:40:58.528509 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.528422    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645"} err="failed to get container status \"1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645\": rpc error: code = NotFound desc = could not find container \"1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645\": container with ID starting with 1822f854f4f403c465166d7d372bf54ab5e3abd7a09f3afafbf8e8bd9c9d4645 not found: ID does not exist"
Jul 10 14:40:58.554218 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.554137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:58.849155 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:40:58.848776    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fccd4159-1f73-4a33-bc34-a81b84f14d15" path="/var/lib/kubelet/pods/fccd4159-1f73-4a33-bc34-a81b84f14d15/volumes"
Jul 10 14:41:01.417106 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.417081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rqg5c"
Jul 10 14:41:01.432018 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.431994    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-zbx64"
Jul 10 14:41:01.474177 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.474142    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn" event={"ID":"be021526-cbe6-4f16-a1ac-50d9c38b12ac","Type":"ContainerStarted","Data":"39f0ec1e0af225aea7f75963c5f6dd43cb687dd2336954faa580b033a08b5fa1"}
Jul 10 14:41:01.474337 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.474299    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:41:01.475847 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.475819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6" event={"ID":"f23f7574-ffe1-474e-b36c-617f367240b0","Type":"ContainerStarted","Data":"b6bd7f4ab64cd1147205d3b513c8e2cd9081ab5f6693cf287e772675b8bf7bc9"}
Jul 10 14:41:01.475955 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.475918    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:41:01.505697 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.505643    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn" podStartSLOduration=1.949363628 podStartE2EDuration="5.505625614s" totalImagesPullingTime="3.556261986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:01.505414472 +0000 UTC m=+359.211735954" watchObservedRunningTime="2026-07-10 14:41:01.505625614 +0000 UTC m=+359.211947085"
Jul 10 14:41:01.565563 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:01.565498    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6" podStartSLOduration=2.334422049 podStartE2EDuration="5.565485507s" totalImagesPullingTime="3.231063458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:01.563403312 +0000 UTC m=+359.269724791" watchObservedRunningTime="2026-07-10 14:41:01.565485507 +0000 UTC m=+359.271806974"
Jul 10 14:41:32.481804 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:32.481774    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dl6t6"
Jul 10 14:41:32.484753 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:32.484736    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mq8jn"
Jul 10 14:41:44.502867 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.502832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:44.622756 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.622718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:44.635242 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.635216    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xtc74"]
Jul 10 14:41:44.635603 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.635590    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fccd4159-1f73-4a33-bc34-a81b84f14d15" containerName="discovery"
Jul 10 14:41:44.635651 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.635606    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fccd4159-1f73-4a33-bc34-a81b84f14d15" containerName="discovery"
Jul 10 14:41:44.635686 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.635659    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fccd4159-1f73-4a33-bc34-a81b84f14d15" containerName="discovery"
Jul 10 14:41:44.640647 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.640624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:44.643586 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.643566    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 14:41:44.643719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.643590    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-g725k\""
Jul 10 14:41:44.644655 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.644637    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 14:41:44.650768 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.650742    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xtc74"]
Jul 10 14:41:44.748957 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.748929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:44.767128 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.767073    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp6kj\" (UniqueName: \"kubernetes.io/projected/6ae9f999-e47a-457e-9f69-e25ebbf40065-kube-api-access-lp6kj\") pod \"kuadrant-operator-catalog-xtc74\" (UID: \"6ae9f999-e47a-457e-9f69-e25ebbf40065\") " pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:44.867602 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.867570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6kj\" (UniqueName: \"kubernetes.io/projected/6ae9f999-e47a-457e-9f69-e25ebbf40065-kube-api-access-lp6kj\") pod \"kuadrant-operator-catalog-xtc74\" (UID: \"6ae9f999-e47a-457e-9f69-e25ebbf40065\") " pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:44.876808 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.876788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6kj\" (UniqueName: \"kubernetes.io/projected/6ae9f999-e47a-457e-9f69-e25ebbf40065-kube-api-access-lp6kj\") pod \"kuadrant-operator-catalog-xtc74\" (UID: \"6ae9f999-e47a-457e-9f69-e25ebbf40065\") " pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:44.953632 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.953606    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:44.992774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:44.992746    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xtc74"]
Jul 10 14:41:45.084240 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.084159    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xtc74"]
Jul 10 14:41:45.086595 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:41:45.086569    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae9f999_e47a_457e_9f69_e25ebbf40065.slice/crio-28ab7f49a9dd789a7c306b15826e1371c719cd1ba811168f1e6bbda4cbba6223 WatchSource:0}: Error finding container 28ab7f49a9dd789a7c306b15826e1371c719cd1ba811168f1e6bbda4cbba6223: Status 404 returned error can't find the container with id 28ab7f49a9dd789a7c306b15826e1371c719cd1ba811168f1e6bbda4cbba6223
Jul 10 14:41:45.199083 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.199052    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rbh9q"]
Jul 10 14:41:45.202036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.202020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:45.212790 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.212769    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rbh9q"]
Jul 10 14:41:45.271377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.271346    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4ww7\" (UniqueName: \"kubernetes.io/projected/e2a3590f-5df2-484b-834d-e5e1636ec674-kube-api-access-s4ww7\") pod \"kuadrant-operator-catalog-rbh9q\" (UID: \"e2a3590f-5df2-484b-834d-e5e1636ec674\") " pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:45.372708 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.372628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4ww7\" (UniqueName: \"kubernetes.io/projected/e2a3590f-5df2-484b-834d-e5e1636ec674-kube-api-access-s4ww7\") pod \"kuadrant-operator-catalog-rbh9q\" (UID: \"e2a3590f-5df2-484b-834d-e5e1636ec674\") " pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:45.390884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.390856    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4ww7\" (UniqueName: \"kubernetes.io/projected/e2a3590f-5df2-484b-834d-e5e1636ec674-kube-api-access-s4ww7\") pod \"kuadrant-operator-catalog-rbh9q\" (UID: \"e2a3590f-5df2-484b-834d-e5e1636ec674\") " pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:45.456886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.456858    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:45.511216 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.511196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:45.641774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.641699    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rbh9q"]
Jul 10 14:41:45.645560 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:41:45.645507    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a3590f_5df2_484b_834d_e5e1636ec674.slice/crio-b8e26493e22f381646d5773a9e88af3992145d34c62a3aab8f819816ca1cd137 WatchSource:0}: Error finding container b8e26493e22f381646d5773a9e88af3992145d34c62a3aab8f819816ca1cd137: Status 404 returned error can't find the container with id b8e26493e22f381646d5773a9e88af3992145d34c62a3aab8f819816ca1cd137
Jul 10 14:41:45.646259 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.646236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:45.663490 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.663464    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q" event={"ID":"e2a3590f-5df2-484b-834d-e5e1636ec674","Type":"ContainerStarted","Data":"b8e26493e22f381646d5773a9e88af3992145d34c62a3aab8f819816ca1cd137"}
Jul 10 14:41:45.664568 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:45.664548    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xtc74" event={"ID":"6ae9f999-e47a-457e-9f69-e25ebbf40065","Type":"ContainerStarted","Data":"28ab7f49a9dd789a7c306b15826e1371c719cd1ba811168f1e6bbda4cbba6223"}
Jul 10 14:41:47.676182 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.676147    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xtc74" event={"ID":"6ae9f999-e47a-457e-9f69-e25ebbf40065","Type":"ContainerStarted","Data":"58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557"}
Jul 10 14:41:47.676690 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.676210    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xtc74" podUID="6ae9f999-e47a-457e-9f69-e25ebbf40065" containerName="registry-server" containerID="cri-o://58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557" gracePeriod=2
Jul 10 14:41:47.677759 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.677733    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q" event={"ID":"e2a3590f-5df2-484b-834d-e5e1636ec674","Type":"ContainerStarted","Data":"43d4f4650c0a2e134f1e515a717cabb9fcf0ab3629672b8559b9244a3e36e37c"}
Jul 10 14:41:47.706075 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.706032    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xtc74" podStartSLOduration=1.609736663 podStartE2EDuration="3.70601978s" totalImagesPullingTime="2.096283117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:47.705233933 +0000 UTC m=+405.411555402" watchObservedRunningTime="2026-07-10 14:41:47.70601978 +0000 UTC m=+405.412341334"
Jul 10 14:41:47.736364 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.736316    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q" podStartSLOduration=1.203290087 podStartE2EDuration="2.736299442s" totalImagesPullingTime="1.533009355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:47.735703868 +0000 UTC m=+405.442025337" watchObservedRunningTime="2026-07-10 14:41:47.736299442 +0000 UTC m=+405.442620916"
Jul 10 14:41:47.912530 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.912497    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:47.994365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.994274    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp6kj\" (UniqueName: \"kubernetes.io/projected/6ae9f999-e47a-457e-9f69-e25ebbf40065-kube-api-access-lp6kj\") pod \"6ae9f999-e47a-457e-9f69-e25ebbf40065\" (UID: \"6ae9f999-e47a-457e-9f69-e25ebbf40065\") "
Jul 10 14:41:47.996377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:47.996355    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae9f999-e47a-457e-9f69-e25ebbf40065-kube-api-access-lp6kj" (OuterVolumeSpecName: "kube-api-access-lp6kj") pod "6ae9f999-e47a-457e-9f69-e25ebbf40065" (UID: "6ae9f999-e47a-457e-9f69-e25ebbf40065"). InnerVolumeSpecName "kube-api-access-lp6kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:41:48.095877 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.095830    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp6kj\" (UniqueName: \"kubernetes.io/projected/6ae9f999-e47a-457e-9f69-e25ebbf40065-kube-api-access-lp6kj\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:41:48.683852 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.683809    2583 generic.go:356] "Generic (PLEG): container finished" podID="6ae9f999-e47a-457e-9f69-e25ebbf40065" containerID="58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557" exitCode=0
Jul 10 14:41:48.684313 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.683867    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xtc74"
Jul 10 14:41:48.684313 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.683899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xtc74" event={"ID":"6ae9f999-e47a-457e-9f69-e25ebbf40065","Type":"ContainerDied","Data":"58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557"}
Jul 10 14:41:48.684313 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.683998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xtc74" event={"ID":"6ae9f999-e47a-457e-9f69-e25ebbf40065","Type":"ContainerDied","Data":"28ab7f49a9dd789a7c306b15826e1371c719cd1ba811168f1e6bbda4cbba6223"}
Jul 10 14:41:48.684313 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.684027    2583 scope.go:117] "RemoveContainer" containerID="58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557"
Jul 10 14:41:48.693411 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.693393    2583 scope.go:117] "RemoveContainer" containerID="58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557"
Jul 10 14:41:48.693678 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:41:48.693660    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557\": container with ID starting with 58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557 not found: ID does not exist" containerID="58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557"
Jul 10 14:41:48.693747 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.693687    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557"} err="failed to get container status \"58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557\": rpc error: code = NotFound desc = could not find container \"58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557\": container with ID starting with 58791efd57a47499dff40ba51fd0b4f33465a06e7b9ee5bed3b245cae9dd1557 not found: ID does not exist"
Jul 10 14:41:48.705448 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.705423    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xtc74"]
Jul 10 14:41:48.709689 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.709668    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xtc74"]
Jul 10 14:41:48.846082 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:48.846050    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae9f999-e47a-457e-9f69-e25ebbf40065" path="/var/lib/kubelet/pods/6ae9f999-e47a-457e-9f69-e25ebbf40065/volumes"
Jul 10 14:41:55.511321 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:55.511288    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:55.511736 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:55.511537    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:55.533962 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:55.533939    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:41:55.735935 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:41:55.735909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-rbh9q"
Jul 10 14:42:18.306954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:18.306920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:21.309405 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.309371    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"]
Jul 10 14:42:21.309822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.309733    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae9f999-e47a-457e-9f69-e25ebbf40065" containerName="registry-server"
Jul 10 14:42:21.309822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.309745    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae9f999-e47a-457e-9f69-e25ebbf40065" containerName="registry-server"
Jul 10 14:42:21.309822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.309813    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae9f999-e47a-457e-9f69-e25ebbf40065" containerName="registry-server"
Jul 10 14:42:21.328261 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.328233    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"]
Jul 10 14:42:21.328391 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.328359    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:21.330886 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.330864    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 10 14:42:21.331991 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.331971    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-92tf5\""
Jul 10 14:42:21.375545 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.375507    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdk8p\" (UniqueName: \"kubernetes.io/projected/fba938ec-4ea9-4596-858d-50490b870d0c-kube-api-access-hdk8p\") pod \"dns-operator-controller-manager-864dd78c4c-rfwjs\" (UID: \"fba938ec-4ea9-4596-858d-50490b870d0c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:21.476738 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.476711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdk8p\" (UniqueName: \"kubernetes.io/projected/fba938ec-4ea9-4596-858d-50490b870d0c-kube-api-access-hdk8p\") pod \"dns-operator-controller-manager-864dd78c4c-rfwjs\" (UID: \"fba938ec-4ea9-4596-858d-50490b870d0c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:21.489955 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.489929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdk8p\" (UniqueName: \"kubernetes.io/projected/fba938ec-4ea9-4596-858d-50490b870d0c-kube-api-access-hdk8p\") pod \"dns-operator-controller-manager-864dd78c4c-rfwjs\" (UID: \"fba938ec-4ea9-4596-858d-50490b870d0c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:21.640772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.640709    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:21.777293 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.777269    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"]
Jul 10 14:42:21.779596 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:21.779570    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba938ec_4ea9_4596_858d_50490b870d0c.slice/crio-7b3d2d1715fe1c5f4c918c61541ec759e83532b03bed0b8a2ea3f53fffcf885b WatchSource:0}: Error finding container 7b3d2d1715fe1c5f4c918c61541ec759e83532b03bed0b8a2ea3f53fffcf885b: Status 404 returned error can't find the container with id 7b3d2d1715fe1c5f4c918c61541ec759e83532b03bed0b8a2ea3f53fffcf885b
Jul 10 14:42:21.828724 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:21.828692    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs" event={"ID":"fba938ec-4ea9-4596-858d-50490b870d0c","Type":"ContainerStarted","Data":"7b3d2d1715fe1c5f4c918c61541ec759e83532b03bed0b8a2ea3f53fffcf885b"}
Jul 10 14:42:22.975939 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:22.975902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:24.201355 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:24.201325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:24.851358 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:24.846272    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:24.851358 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:24.846344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs" event={"ID":"fba938ec-4ea9-4596-858d-50490b870d0c","Type":"ContainerStarted","Data":"69f46ce21b7feffdbde02523263d5a947ee31d443d163ea3778fcf7854a04285"}
Jul 10 14:42:25.092027 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.091970    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs" podStartSLOduration=2.015037687 podStartE2EDuration="4.091950034s" totalImagesPullingTime="2.076912347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:24.889648121 +0000 UTC m=+442.595969590" watchObservedRunningTime="2026-07-10 14:42:25.091950034 +0000 UTC m=+442.798271505"
Jul 10 14:42:25.093822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.093794    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"]
Jul 10 14:42:25.095963 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.095944    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:25.098772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.098754    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t4nzk\""
Jul 10 14:42:25.107856 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.107837    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8thgh\" (UniqueName: \"kubernetes.io/projected/4a4c71fd-b586-4c86-abe0-fd000c9959a4-kube-api-access-8thgh\") pod \"limitador-operator-controller-manager-859d7b78bd-f5pch\" (UID: \"4a4c71fd-b586-4c86-abe0-fd000c9959a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:25.112474 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.112452    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"]
Jul 10 14:42:25.208566 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.208510    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8thgh\" (UniqueName: \"kubernetes.io/projected/4a4c71fd-b586-4c86-abe0-fd000c9959a4-kube-api-access-8thgh\") pod \"limitador-operator-controller-manager-859d7b78bd-f5pch\" (UID: \"4a4c71fd-b586-4c86-abe0-fd000c9959a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:25.220668 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.220639    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8thgh\" (UniqueName: \"kubernetes.io/projected/4a4c71fd-b586-4c86-abe0-fd000c9959a4-kube-api-access-8thgh\") pod \"limitador-operator-controller-manager-859d7b78bd-f5pch\" (UID: \"4a4c71fd-b586-4c86-abe0-fd000c9959a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:25.405703 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.405595    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:25.469082 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.469045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:25.546375 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.546351    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"]
Jul 10 14:42:25.548376 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:25.548336    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4c71fd_b586_4c86_abe0_fd000c9959a4.slice/crio-e6f7c29546eb6f69e899b5622cde4608ed2f555901261da9c4ea927a5c636abc WatchSource:0}: Error finding container e6f7c29546eb6f69e899b5622cde4608ed2f555901261da9c4ea927a5c636abc: Status 404 returned error can't find the container with id e6f7c29546eb6f69e899b5622cde4608ed2f555901261da9c4ea927a5c636abc
Jul 10 14:42:25.851788 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:25.851754    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch" event={"ID":"4a4c71fd-b586-4c86-abe0-fd000c9959a4","Type":"ContainerStarted","Data":"e6f7c29546eb6f69e899b5622cde4608ed2f555901261da9c4ea927a5c636abc"}
Jul 10 14:42:26.065240 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:26.065209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:26.382071 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:26.382038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:27.548044 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:27.548013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:27.641105 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:27.641074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:27.862475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:27.862397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch" event={"ID":"4a4c71fd-b586-4c86-abe0-fd000c9959a4","Type":"ContainerStarted","Data":"8f6b384417b3090a0c5e6914cc776a048a860878bf3c1d501bd793fa98ddbfed"}
Jul 10 14:42:27.862625 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:27.862512    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:27.882477 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:27.882408    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch" podStartSLOduration=1.288873285 podStartE2EDuration="2.882392884s" totalImagesPullingTime="1.593519599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:27.8813901 +0000 UTC m=+445.587711565" watchObservedRunningTime="2026-07-10 14:42:27.882392884 +0000 UTC m=+445.588714407"
Jul 10 14:42:28.977653 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:28.977625    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"]
Jul 10 14:42:28.980939 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:28.980922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:28.983769 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:28.983753    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-48krx\""
Jul 10 14:42:28.999892 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:28.999868    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"]
Jul 10 14:42:29.041039 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.041016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq9j6\" (UniqueName: \"kubernetes.io/projected/7a696a6b-9f3b-4d25-b663-83e67a2636ad-kube-api-access-hq9j6\") pod \"authorino-operator-75cc8d5cb7-7cd7q\" (UID: \"7a696a6b-9f3b-4d25-b663-83e67a2636ad\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:29.141596 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.141567    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq9j6\" (UniqueName: \"kubernetes.io/projected/7a696a6b-9f3b-4d25-b663-83e67a2636ad-kube-api-access-hq9j6\") pod \"authorino-operator-75cc8d5cb7-7cd7q\" (UID: \"7a696a6b-9f3b-4d25-b663-83e67a2636ad\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:29.154082 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.154051    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq9j6\" (UniqueName: \"kubernetes.io/projected/7a696a6b-9f3b-4d25-b663-83e67a2636ad-kube-api-access-hq9j6\") pod \"authorino-operator-75cc8d5cb7-7cd7q\" (UID: \"7a696a6b-9f3b-4d25-b663-83e67a2636ad\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:29.290728 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.290653    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:29.438696 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.438660    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"]
Jul 10 14:42:29.441042 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:29.441013    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a696a6b_9f3b_4d25_b663_83e67a2636ad.slice/crio-78e0ccdd98d99c876e7b6622100df33f88acd16cf76d52b48bd12e6414deaff5 WatchSource:0}: Error finding container 78e0ccdd98d99c876e7b6622100df33f88acd16cf76d52b48bd12e6414deaff5: Status 404 returned error can't find the container with id 78e0ccdd98d99c876e7b6622100df33f88acd16cf76d52b48bd12e6414deaff5
Jul 10 14:42:29.497501 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.497478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:29.871814 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:29.871780    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q" event={"ID":"7a696a6b-9f3b-4d25-b663-83e67a2636ad","Type":"ContainerStarted","Data":"78e0ccdd98d99c876e7b6622100df33f88acd16cf76d52b48bd12e6414deaff5"}
Jul 10 14:42:31.238353 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:31.238313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:32.891839 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:32.891802    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q" event={"ID":"7a696a6b-9f3b-4d25-b663-83e67a2636ad","Type":"ContainerStarted","Data":"cf5179f8394d894c771746fc79e93bfc8127c4c9f2d26c921491adf94fc68a74"}
Jul 10 14:42:32.892236 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:32.891859    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:32.938193 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:32.938141    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q" podStartSLOduration=2.467890861 podStartE2EDuration="4.938126632s" totalImagesPullingTime="2.470235771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:32.937138315 +0000 UTC m=+450.643459815" watchObservedRunningTime="2026-07-10 14:42:32.938126632 +0000 UTC m=+450.644448100"
Jul 10 14:42:32.998400 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:32.998374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:34.004695 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.004668    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"]
Jul 10 14:42:34.008119 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.008101    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.010838 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.010812    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fp77v\""
Jul 10 14:42:34.020013 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.019989    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"]
Jul 10 14:42:34.086062 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.086032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkzc7\" (UniqueName: \"kubernetes.io/projected/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-kube-api-access-jkzc7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.086223 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.086080    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.186787 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.186749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.186988 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.186818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkzc7\" (UniqueName: \"kubernetes.io/projected/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-kube-api-access-jkzc7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.187181 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.187158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.195911 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.195884    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkzc7\" (UniqueName: \"kubernetes.io/projected/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-kube-api-access-jkzc7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.319040 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.318956    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:34.475946 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.475914    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"]
Jul 10 14:42:34.479610 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:34.479578    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae2612d_caaa_4fbd_a03a_c0dc8d3a1d9d.slice/crio-158c03f25ba41d24a9f60908758a99df2201aff59384696e55bb2add7b224985 WatchSource:0}: Error finding container 158c03f25ba41d24a9f60908758a99df2201aff59384696e55bb2add7b224985: Status 404 returned error can't find the container with id 158c03f25ba41d24a9f60908758a99df2201aff59384696e55bb2add7b224985
Jul 10 14:42:34.696246 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.696203    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:34.902489 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:34.902453    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4" event={"ID":"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d","Type":"ContainerStarted","Data":"158c03f25ba41d24a9f60908758a99df2201aff59384696e55bb2add7b224985"}
Jul 10 14:42:35.854634 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:35.854606    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rfwjs"
Jul 10 14:42:36.148752 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:36.148655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:36.331956 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:36.331916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:38.868272 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:38.868243    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:39.725587 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:39.725539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:39.870130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:39.870091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:40.937162 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:40.937132    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4" event={"ID":"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d","Type":"ContainerStarted","Data":"021004cfa15430bb64283346755ed829e9746aafa8123d42d33b5198afe54585"}
Jul 10 14:42:40.937582 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:40.937202    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:43.167816 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:43.167762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:43.275753 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:43.275706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:43.716432 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:43.716396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:43.812017 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:43.811967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:43.899181 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:43.899154    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7cd7q"
Jul 10 14:42:43.918834 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:43.918790    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4" podStartSLOduration=4.8815743099999995 podStartE2EDuration="10.918772851s" totalImagesPullingTime="6.037198541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:40.972422918 +0000 UTC m=+458.678744386" watchObservedRunningTime="2026-07-10 14:42:43.918772851 +0000 UTC m=+461.625094319"
Jul 10 14:42:47.398817 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:47.398741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:47.521164 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:47.521132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:51.944333 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:51.944303    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:52.920077 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:52.920041    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"]
Jul 10 14:42:52.927999 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:52.927967    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:52.946482 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:52.946457    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"]
Jul 10 14:42:53.043996 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.043961    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd466\" (UniqueName: \"kubernetes.io/projected/9c930fa8-8100-4d1c-9982-cffbc4091c8c-kube-api-access-qd466\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.044133 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.044022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c930fa8-8100-4d1c-9982-cffbc4091c8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.145423 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.145395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c930fa8-8100-4d1c-9982-cffbc4091c8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.145593 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.145451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd466\" (UniqueName: \"kubernetes.io/projected/9c930fa8-8100-4d1c-9982-cffbc4091c8c-kube-api-access-qd466\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.145758 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.145741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c930fa8-8100-4d1c-9982-cffbc4091c8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.162759 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.162726    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd466\" (UniqueName: \"kubernetes.io/projected/9c930fa8-8100-4d1c-9982-cffbc4091c8c-kube-api-access-qd466\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.237976 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.237913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:53.379611 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.379564    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"]
Jul 10 14:42:53.382505 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:53.382479    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c930fa8_8100_4d1c_9982_cffbc4091c8c.slice/crio-8f9dd7775488f05ee111e87e437a577a11a59dc633c300f9378d2a122d1572e2 WatchSource:0}: Error finding container 8f9dd7775488f05ee111e87e437a577a11a59dc633c300f9378d2a122d1572e2: Status 404 returned error can't find the container with id 8f9dd7775488f05ee111e87e437a577a11a59dc633c300f9378d2a122d1572e2
Jul 10 14:42:53.815431 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.815397    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"]
Jul 10 14:42:53.822974 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.822941    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"]
Jul 10 14:42:53.839292 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.839263    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"]
Jul 10 14:42:53.839459 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.839438    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" containerName="manager" containerID="cri-o://021004cfa15430bb64283346755ed829e9746aafa8123d42d33b5198afe54585" gracePeriod=2
Jul 10 14:42:53.843245 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.843198    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"]
Jul 10 14:42:53.843731 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.843711    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" containerName="manager"
Jul 10 14:42:53.843852 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.843734    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" containerName="manager"
Jul 10 14:42:53.843914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.843859    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" containerName="manager"
Jul 10 14:42:53.847556 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.847512    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:53.852872 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.852850    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"]
Jul 10 14:42:53.860463 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:53.860430    2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\": pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:53.862672 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:53.862638    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:53.863239 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.863215    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"]
Jul 10 14:42:53.866190 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.866164    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"]
Jul 10 14:42:53.866541 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.866503    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" containerName="manager"
Jul 10 14:42:53.866541 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.866533    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" containerName="manager"
Jul 10 14:42:53.866660 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.866619    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" containerName="manager"
Jul 10 14:42:53.870167 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.870146    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"]
Jul 10 14:42:53.870289 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.870271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:53.870354 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.870321    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" containerName="manager" containerID="cri-o://8f6b384417b3090a0c5e6914cc776a048a860878bf3c1d501bd793fa98ddbfed" gracePeriod=2
Jul 10 14:42:53.874040 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.874016    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"]
Jul 10 14:42:53.891217 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.891192    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"]
Jul 10 14:42:53.892695 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:53.892669    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:53.905484 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.905455    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"]
Jul 10 14:42:53.906065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.906044    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" containerName="manager"
Jul 10 14:42:53.906144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.906068    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" containerName="manager"
Jul 10 14:42:53.906212 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.906204    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" containerName="manager"
Jul 10 14:42:53.909819 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.909797    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:42:53.920344 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:53.920320    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f5pch\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:53.922686 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:53.922657    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:53.923164 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.923148    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"]
Jul 10 14:42:53.925136 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:53.925106    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f5pch\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:53.951673 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.951650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxzqh\" (UniqueName: \"kubernetes.io/projected/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-kube-api-access-bxzqh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mjlqx\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:53.952039 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.951741    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44rgt\" (UniqueName: \"kubernetes.io/projected/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-kube-api-access-44rgt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pfvk\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:53.952039 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.951816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mjlqx\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:53.952039 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.951951    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pfvk\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:53.952039 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:53.951961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:54.007720 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.007693    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx" container="manager"
Jul 10 14:42:54.009973 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.009939    2583 generic.go:356] "Generic (PLEG): container finished" podID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" containerID="8f6b384417b3090a0c5e6914cc776a048a860878bf3c1d501bd793fa98ddbfed" exitCode=0
Jul 10 14:42:54.012443 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.012410    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f5pch\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:54.012741 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.012718    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" containerID="021004cfa15430bb64283346755ed829e9746aafa8123d42d33b5198afe54585" exitCode=0
Jul 10 14:42:54.014629 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.014599    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:54.016629 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.016607    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:54.040761 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.040732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:54.053180 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxzqh\" (UniqueName: \"kubernetes.io/projected/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-kube-api-access-bxzqh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mjlqx\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:54.053324 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nfz5\" (UniqueName: \"kubernetes.io/projected/26e00d4a-b2af-4e38-8db8-72763b0e066f-kube-api-access-4nfz5\") pod \"limitador-operator-controller-manager-859d7b78bd-blxxm\" (UID: \"26e00d4a-b2af-4e38-8db8-72763b0e066f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:42:54.053324 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44rgt\" (UniqueName: \"kubernetes.io/projected/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-kube-api-access-44rgt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pfvk\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:54.053324 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053248    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mjlqx\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:54.053324 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pfvk\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:54.053841 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053624    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pfvk\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:54.053976 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.053897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mjlqx\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:54.067767 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.067705    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44rgt\" (UniqueName: \"kubernetes.io/projected/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-kube-api-access-44rgt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6pfvk\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:54.069033 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.069010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxzqh\" (UniqueName: \"kubernetes.io/projected/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-kube-api-access-bxzqh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mjlqx\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:54.112204 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.112182    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:54.114859 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.114831    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f5pch\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:54.117024 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.116999    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:54.120000 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.119975    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:54.122777 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.122760    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:54.124973 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.124951    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f5pch\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:54.127278 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.127252    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:54.131376 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:54.131359    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pxpx4\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:54.153767 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.153748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nfz5\" (UniqueName: \"kubernetes.io/projected/26e00d4a-b2af-4e38-8db8-72763b0e066f-kube-api-access-4nfz5\") pod \"limitador-operator-controller-manager-859d7b78bd-blxxm\" (UID: \"26e00d4a-b2af-4e38-8db8-72763b0e066f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:42:54.167354 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.167334    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nfz5\" (UniqueName: \"kubernetes.io/projected/26e00d4a-b2af-4e38-8db8-72763b0e066f-kube-api-access-4nfz5\") pod \"limitador-operator-controller-manager-859d7b78bd-blxxm\" (UID: \"26e00d4a-b2af-4e38-8db8-72763b0e066f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:42:54.254027 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.254002    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-extensions-socket-volume\") pod \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") "
Jul 10 14:42:54.254156 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.254066    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8thgh\" (UniqueName: \"kubernetes.io/projected/4a4c71fd-b586-4c86-abe0-fd000c9959a4-kube-api-access-8thgh\") pod \"4a4c71fd-b586-4c86-abe0-fd000c9959a4\" (UID: \"4a4c71fd-b586-4c86-abe0-fd000c9959a4\") "
Jul 10 14:42:54.254156 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.254125    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkzc7\" (UniqueName: \"kubernetes.io/projected/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-kube-api-access-jkzc7\") pod \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\" (UID: \"5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d\") "
Jul 10 14:42:54.254430 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.254410    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" (UID: "5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:42:54.256054 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.256032    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-kube-api-access-jkzc7" (OuterVolumeSpecName: "kube-api-access-jkzc7") pod "5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" (UID: "5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d"). InnerVolumeSpecName "kube-api-access-jkzc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:42:54.256137 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.256064    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4c71fd-b586-4c86-abe0-fd000c9959a4-kube-api-access-8thgh" (OuterVolumeSpecName: "kube-api-access-8thgh") pod "4a4c71fd-b586-4c86-abe0-fd000c9959a4" (UID: "4a4c71fd-b586-4c86-abe0-fd000c9959a4"). InnerVolumeSpecName "kube-api-access-8thgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:42:54.263199 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.263177    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:54.269142 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.269126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:54.275053 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.275027    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:42:54.355614 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.355499    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkzc7\" (UniqueName: \"kubernetes.io/projected/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-kube-api-access-jkzc7\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:42:54.355768 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.355615    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d-extensions-socket-volume\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:42:54.355768 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.355634    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8thgh\" (UniqueName: \"kubernetes.io/projected/4a4c71fd-b586-4c86-abe0-fd000c9959a4-kube-api-access-8thgh\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:42:54.437383 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.437350    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"]
Jul 10 14:42:54.441085 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:54.441059    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2f097c_0574_4c95_b5dc_2767bbcfbd4a.slice/crio-355bec1f323c7ac8aa295ebbb602cfaba7dd9a7f670f04650c5cf1d8645d0cfd WatchSource:0}: Error finding container 355bec1f323c7ac8aa295ebbb602cfaba7dd9a7f670f04650c5cf1d8645d0cfd: Status 404 returned error can't find the container with id 355bec1f323c7ac8aa295ebbb602cfaba7dd9a7f670f04650c5cf1d8645d0cfd
Jul 10 14:42:54.683561 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.683496    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"]
Jul 10 14:42:54.685080 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.685059    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"]
Jul 10 14:42:54.686408 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:54.686381    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4e3bf6_50b3_4bdc_be5a_a422338b15d1.slice/crio-5c419b3037ac96b306ebb52f9fe9ff24962368d70f46fc1b78d788089ae052f0 WatchSource:0}: Error finding container 5c419b3037ac96b306ebb52f9fe9ff24962368d70f46fc1b78d788089ae052f0: Status 404 returned error can't find the container with id 5c419b3037ac96b306ebb52f9fe9ff24962368d70f46fc1b78d788089ae052f0
Jul 10 14:42:54.687031 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:42:54.687014    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e00d4a_b2af_4e38_8db8_72763b0e066f.slice/crio-75b2f5aeb0d99d2621c1658952987447db6f09bb147884c64a10900ec594d9ed WatchSource:0}: Error finding container 75b2f5aeb0d99d2621c1658952987447db6f09bb147884c64a10900ec594d9ed: Status 404 returned error can't find the container with id 75b2f5aeb0d99d2621c1658952987447db6f09bb147884c64a10900ec594d9ed
Jul 10 14:42:54.849033 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.849005    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a4c71fd-b586-4c86-abe0-fd000c9959a4" path="/var/lib/kubelet/pods/4a4c71fd-b586-4c86-abe0-fd000c9959a4/volumes"
Jul 10 14:42:54.849399 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:54.849386    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d" path="/var/lib/kubelet/pods/5ae2612d-caaa-4fbd-a03a-c0dc8d3a1d9d/volumes"
Jul 10 14:42:55.019856 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.019740    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" containerName="manager" containerID="cri-o://f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17" gracePeriod=2
Jul 10 14:42:55.021309 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.021280    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm" event={"ID":"26e00d4a-b2af-4e38-8db8-72763b0e066f","Type":"ContainerStarted","Data":"a7bdd8368d9b84b90d8dabb03276abafc96f4c8c52da8d23c1b7d7386f308b44"}
Jul 10 14:42:55.021433 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.021371    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:42:55.021433 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.021392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm" event={"ID":"26e00d4a-b2af-4e38-8db8-72763b0e066f","Type":"ContainerStarted","Data":"75b2f5aeb0d99d2621c1658952987447db6f09bb147884c64a10900ec594d9ed"}
Jul 10 14:42:55.023098 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.023072    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx" event={"ID":"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a","Type":"ContainerStarted","Data":"6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3"}
Jul 10 14:42:55.023210 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.023111    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx" event={"ID":"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a","Type":"ContainerStarted","Data":"355bec1f323c7ac8aa295ebbb602cfaba7dd9a7f670f04650c5cf1d8645d0cfd"}
Jul 10 14:42:55.023210 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.023158    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:42:55.024922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.024891    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk" event={"ID":"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1","Type":"ContainerStarted","Data":"9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475"}
Jul 10 14:42:55.025035 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.024949    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk" event={"ID":"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1","Type":"ContainerStarted","Data":"5c419b3037ac96b306ebb52f9fe9ff24962368d70f46fc1b78d788089ae052f0"}
Jul 10 14:42:55.025035 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.024980    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:42:55.026397 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.026370    2583 scope.go:117] "RemoveContainer" containerID="8f6b384417b3090a0c5e6914cc776a048a860878bf3c1d501bd793fa98ddbfed"
Jul 10 14:42:55.026486 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.026411    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f5pch"
Jul 10 14:42:55.027676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.027656    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pxpx4"
Jul 10 14:42:55.036335 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.036298    2583 scope.go:117] "RemoveContainer" containerID="021004cfa15430bb64283346755ed829e9746aafa8123d42d33b5198afe54585"
Jul 10 14:42:55.045811 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.045767    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm" podStartSLOduration=2.045753939 podStartE2EDuration="2.045753939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:55.043943275 +0000 UTC m=+472.750264744" watchObservedRunningTime="2026-07-10 14:42:55.045753939 +0000 UTC m=+472.752075481"
Jul 10 14:42:55.070592 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.070548    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx" podStartSLOduration=2.070531202 podStartE2EDuration="2.070531202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:55.070016331 +0000 UTC m=+472.776337800" watchObservedRunningTime="2026-07-10 14:42:55.070531202 +0000 UTC m=+472.776852663"
Jul 10 14:42:55.100844 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.100804    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk" podStartSLOduration=2.100791545 podStartE2EDuration="2.100791545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:42:55.099053272 +0000 UTC m=+472.805374753" watchObservedRunningTime="2026-07-10 14:42:55.100791545 +0000 UTC m=+472.807113014"
Jul 10 14:42:55.271547 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.271474    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:55.273820 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:55.273792    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:55.366025 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.365999    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c930fa8-8100-4d1c-9982-cffbc4091c8c-extensions-socket-volume\") pod \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") "
Jul 10 14:42:55.366148 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.366047    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd466\" (UniqueName: \"kubernetes.io/projected/9c930fa8-8100-4d1c-9982-cffbc4091c8c-kube-api-access-qd466\") pod \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\" (UID: \"9c930fa8-8100-4d1c-9982-cffbc4091c8c\") "
Jul 10 14:42:55.366400 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.366377    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c930fa8-8100-4d1c-9982-cffbc4091c8c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9c930fa8-8100-4d1c-9982-cffbc4091c8c" (UID: "9c930fa8-8100-4d1c-9982-cffbc4091c8c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:42:55.368069 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.368051    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c930fa8-8100-4d1c-9982-cffbc4091c8c-kube-api-access-qd466" (OuterVolumeSpecName: "kube-api-access-qd466") pod "9c930fa8-8100-4d1c-9982-cffbc4091c8c" (UID: "9c930fa8-8100-4d1c-9982-cffbc4091c8c"). InnerVolumeSpecName "kube-api-access-qd466". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:42:55.467627 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.467600    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c930fa8-8100-4d1c-9982-cffbc4091c8c-extensions-socket-volume\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:42:55.467627 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:55.467625    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd466\" (UniqueName: \"kubernetes.io/projected/9c930fa8-8100-4d1c-9982-cffbc4091c8c-kube-api-access-qd466\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:42:56.036183 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.036145    2583 generic.go:356] "Generic (PLEG): container finished" podID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" containerID="f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17" exitCode=0
Jul 10 14:42:56.036641 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.036190    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:56.036641 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.036235    2583 scope.go:117] "RemoveContainer" containerID="f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17"
Jul 10 14:42:56.038779 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:56.038748    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:56.047107 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.047095    2583 scope.go:117] "RemoveContainer" containerID="f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17"
Jul 10 14:42:56.047174 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:56.047146    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8mbmx\" is forbidden: User \"system:node:ip-10-0-136-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-210.ec2.internal' and this object" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8mbmx"
Jul 10 14:42:56.047590 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:42:56.047562    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17\": container with ID starting with f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17 not found: ID does not exist" containerID="f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17"
Jul 10 14:42:56.047651 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.047602    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17"} err="failed to get container status \"f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17\": rpc error: code = NotFound desc = could not find container \"f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17\": container with ID starting with f7e731c81f6e64a0789f615e4196344c6c0278904c279e5f123c8a176ad01f17 not found: ID does not exist"
Jul 10 14:42:56.662003 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.661970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:56.777163 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.777130    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:56.847186 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.847158    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c930fa8-8100-4d1c-9982-cffbc4091c8c" path="/var/lib/kubelet/pods/9c930fa8-8100-4d1c-9982-cffbc4091c8c/volumes"
Jul 10 14:42:56.877865 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:42:56.877840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:01.574763 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:01.574729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:01.657716 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:01.657685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:04.805381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:04.805347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:04.882180 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:04.882141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:05.092192 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:05.092120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:05.154796 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:05.154766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:06.039112 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.039082    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blxxm"
Jul 10 14:43:06.039474 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.039151    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:43:06.039474 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.039181    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:43:06.131511 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.131481    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"]
Jul 10 14:43:06.131743 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.131715    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk" podUID="cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" containerName="manager" containerID="cri-o://9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475" gracePeriod=10
Jul 10 14:43:06.376689 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.376665    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:43:06.455651 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.455627    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-extensions-socket-volume\") pod \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") "
Jul 10 14:43:06.455811 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.455674    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44rgt\" (UniqueName: \"kubernetes.io/projected/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-kube-api-access-44rgt\") pod \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\" (UID: \"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1\") "
Jul 10 14:43:06.455977 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.455953    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" (UID: "cb4e3bf6-50b3-4bdc-be5a-a422338b15d1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:43:06.457493 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.457471    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-kube-api-access-44rgt" (OuterVolumeSpecName: "kube-api-access-44rgt") pod "cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" (UID: "cb4e3bf6-50b3-4bdc-be5a-a422338b15d1"). InnerVolumeSpecName "kube-api-access-44rgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:43:06.557177 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.557115    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-extensions-socket-volume\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:43:06.557177 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:06.557140    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44rgt\" (UniqueName: \"kubernetes.io/projected/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1-kube-api-access-44rgt\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:43:07.095260 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.095225    2583 generic.go:356] "Generic (PLEG): container finished" podID="cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" containerID="9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475" exitCode=0
Jul 10 14:43:07.095640 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.095269    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"
Jul 10 14:43:07.095640 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.095312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk" event={"ID":"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1","Type":"ContainerDied","Data":"9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475"}
Jul 10 14:43:07.095640 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.095362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk" event={"ID":"cb4e3bf6-50b3-4bdc-be5a-a422338b15d1","Type":"ContainerDied","Data":"5c419b3037ac96b306ebb52f9fe9ff24962368d70f46fc1b78d788089ae052f0"}
Jul 10 14:43:07.095640 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.095383    2583 scope.go:117] "RemoveContainer" containerID="9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475"
Jul 10 14:43:07.103845 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.103828    2583 scope.go:117] "RemoveContainer" containerID="9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475"
Jul 10 14:43:07.104101 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:43:07.104081    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475\": container with ID starting with 9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475 not found: ID does not exist" containerID="9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475"
Jul 10 14:43:07.104193 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.104116    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475"} err="failed to get container status \"9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475\": rpc error: code = NotFound desc = could not find container \"9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475\": container with ID starting with 9e48503a9344451391cbf062f1c92b85f8f2458e048c0c2edba8e09ef1e20475 not found: ID does not exist"
Jul 10 14:43:07.121217 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.121192    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"]
Jul 10 14:43:07.127875 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:07.127855    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6pfvk"]
Jul 10 14:43:08.845964 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:08.845934    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" path="/var/lib/kubelet/pods/cb4e3bf6-50b3-4bdc-be5a-a422338b15d1/volumes"
Jul 10 14:43:11.783419 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:11.783384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:11.883680 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:11.883641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:14.992970 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:14.992938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:15.088949 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:15.088919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:19.967883 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:19.967849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:20.068354 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:20.068327    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:20.571435 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:20.571405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:20.639415 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:20.639389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:22.476414 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.476383    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"]
Jul 10 14:43:22.476789 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.476762    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" containerName="manager"
Jul 10 14:43:22.476789 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.476774    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" containerName="manager"
Jul 10 14:43:22.476876 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.476866    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb4e3bf6-50b3-4bdc-be5a-a422338b15d1" containerName="manager"
Jul 10 14:43:22.479635 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.479614    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.482250 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.482229    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tpjks\""
Jul 10 14:43:22.482414 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.482397    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 14:43:22.491772 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.491750    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"]
Jul 10 14:43:22.494372 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494461 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494461 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494452    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494475    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jh8b\" (UniqueName: \"kubernetes.io/projected/e0f16b21-14c2-4081-b5a8-6699f5840a7b-kube-api-access-4jh8b\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494597 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494575    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494798 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.494798 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.494626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.500101 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.500076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:22.595661 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595661 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595709    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595769    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jh8b\" (UniqueName: \"kubernetes.io/projected/e0f16b21-14c2-4081-b5a8-6699f5840a7b-kube-api-access-4jh8b\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595788    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.595899 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.595857    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.596263 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.596159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.596320 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.596297    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.596360 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.596340    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.596437 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.596415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.596511 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.596496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.598643 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.598616    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.598798 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.598782    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.605601 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.605580    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f16b21-14c2-4081-b5a8-6699f5840a7b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.605790 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.605770    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jh8b\" (UniqueName: \"kubernetes.io/projected/e0f16b21-14c2-4081-b5a8-6699f5840a7b-kube-api-access-4jh8b\") pod \"maas-default-gateway-openshift-default-56654dcf7-pzt5b\" (UID: \"e0f16b21-14c2-4081-b5a8-6699f5840a7b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.791252 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.791165    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:22.934179 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.934138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:22.948925 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:22.948903    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"]
Jul 10 14:43:22.951354 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:43:22.951334    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f16b21_14c2_4081_b5a8_6699f5840a7b.slice/crio-032414e9ac578cdca61e6116db71d7f4543ef3ab4a5b0318724dd41f8b20ba05 WatchSource:0}: Error finding container 032414e9ac578cdca61e6116db71d7f4543ef3ab4a5b0318724dd41f8b20ba05: Status 404 returned error can't find the container with id 032414e9ac578cdca61e6116db71d7f4543ef3ab4a5b0318724dd41f8b20ba05
Jul 10 14:43:23.173879 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.173841    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" event={"ID":"e0f16b21-14c2-4081-b5a8-6699f5840a7b","Type":"ContainerStarted","Data":"032414e9ac578cdca61e6116db71d7f4543ef3ab4a5b0318724dd41f8b20ba05"}
Jul 10 14:43:23.424719 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.424638    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"]
Jul 10 14:43:23.424860 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.424828    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx" podUID="ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" containerName="manager" containerID="cri-o://6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3" gracePeriod=10
Jul 10 14:43:23.671604 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.671579    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:43:23.704800 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.704731    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-extensions-socket-volume\") pod \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") "
Jul 10 14:43:23.704929 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.704804    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxzqh\" (UniqueName: \"kubernetes.io/projected/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-kube-api-access-bxzqh\") pod \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\" (UID: \"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a\") "
Jul 10 14:43:23.705126 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.705096    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" (UID: "ba2f097c-0574-4c95-b5dc-2767bbcfbd4a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:43:23.706870 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.706845    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-kube-api-access-bxzqh" (OuterVolumeSpecName: "kube-api-access-bxzqh") pod "ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" (UID: "ba2f097c-0574-4c95-b5dc-2767bbcfbd4a"). InnerVolumeSpecName "kube-api-access-bxzqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:43:23.806117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.806090    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-extensions-socket-volume\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:43:23.806117 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:23.806114    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxzqh\" (UniqueName: \"kubernetes.io/projected/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a-kube-api-access-bxzqh\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:43:24.180074 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.180040    2583 generic.go:356] "Generic (PLEG): container finished" podID="ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" containerID="6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3" exitCode=0
Jul 10 14:43:24.180248 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.180089    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"
Jul 10 14:43:24.180248 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.180118    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx" event={"ID":"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a","Type":"ContainerDied","Data":"6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3"}
Jul 10 14:43:24.180248 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.180164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx" event={"ID":"ba2f097c-0574-4c95-b5dc-2767bbcfbd4a","Type":"ContainerDied","Data":"355bec1f323c7ac8aa295ebbb602cfaba7dd9a7f670f04650c5cf1d8645d0cfd"}
Jul 10 14:43:24.180248 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.180188    2583 scope.go:117] "RemoveContainer" containerID="6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3"
Jul 10 14:43:24.191322 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.191300    2583 scope.go:117] "RemoveContainer" containerID="6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3"
Jul 10 14:43:24.191702 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:43:24.191682    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3\": container with ID starting with 6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3 not found: ID does not exist" containerID="6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3"
Jul 10 14:43:24.191777 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.191711    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3"} err="failed to get container status \"6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3\": rpc error: code = NotFound desc = could not find container \"6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3\": container with ID starting with 6d0278ed6c1ba2e3abb216b33c76a9a7a433893d43d3bba59ded4361ebbdffb3 not found: ID does not exist"
Jul 10 14:43:24.204195 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.204171    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"]
Jul 10 14:43:24.212769 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.212749    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mjlqx"]
Jul 10 14:43:24.846739 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.846709    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" path="/var/lib/kubelet/pods/ba2f097c-0574-4c95-b5dc-2767bbcfbd4a/volumes"
Jul 10 14:43:24.901764 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:24.901727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:25.039207 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:25.039168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:26.432554 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:26.432495    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:43:26.432850 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:26.432587    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:43:26.432850 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:26.432616    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:43:27.200015 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:27.199981    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" event={"ID":"e0f16b21-14c2-4081-b5a8-6699f5840a7b","Type":"ContainerStarted","Data":"60a0bbfec1a6fe975a72f01e53e94e015336714e66987020586372ef39e58012"}
Jul 10 14:43:27.224238 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:27.224181    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" podStartSLOduration=1.745573131 podStartE2EDuration="5.224161176s" totalImagesPullingTime="3.478588045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:43:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:43:27.221867833 +0000 UTC m=+504.928189319" watchObservedRunningTime="2026-07-10 14:43:27.224161176 +0000 UTC m=+504.930482646"
Jul 10 14:43:27.792144 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:27.792109    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:27.796627 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:27.796602    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:28.204490 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:28.204463    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:28.205445 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:28.205424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:43:42.048851 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.048817    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:42.049358 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.049248    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" containerName="manager"
Jul 10 14:43:42.049358 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.049260    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" containerName="manager"
Jul 10 14:43:42.049358 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.049339    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba2f097c-0574-4c95-b5dc-2767bbcfbd4a" containerName="manager"
Jul 10 14:43:42.051974 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.051952    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.054774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.054748    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 10 14:43:42.054863 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.054762    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-28lrd\""
Jul 10 14:43:42.061984 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.061961    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:42.135323 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.135292    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:42.155506 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.155483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19031033-a02e-4c08-9c47-57a5cd70acc2-config-file\") pod \"limitador-limitador-64c94fd64f-jjbh5\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.155649 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.155511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp8mc\" (UniqueName: \"kubernetes.io/projected/19031033-a02e-4c08-9c47-57a5cd70acc2-kube-api-access-bp8mc\") pod \"limitador-limitador-64c94fd64f-jjbh5\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.255962 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.255928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19031033-a02e-4c08-9c47-57a5cd70acc2-config-file\") pod \"limitador-limitador-64c94fd64f-jjbh5\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.255962 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.255966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp8mc\" (UniqueName: \"kubernetes.io/projected/19031033-a02e-4c08-9c47-57a5cd70acc2-kube-api-access-bp8mc\") pod \"limitador-limitador-64c94fd64f-jjbh5\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.256606 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.256587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19031033-a02e-4c08-9c47-57a5cd70acc2-config-file\") pod \"limitador-limitador-64c94fd64f-jjbh5\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.268030 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.267998    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp8mc\" (UniqueName: \"kubernetes.io/projected/19031033-a02e-4c08-9c47-57a5cd70acc2-kube-api-access-bp8mc\") pod \"limitador-limitador-64c94fd64f-jjbh5\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.340465 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.340396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:42.364401 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.364379    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:42.417152 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.417109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:42.462261 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.462231    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:43:42.465161 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.465137    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.475952 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.475931    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:43:42.502142 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.502119    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:42.505740 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:43:42.505714    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19031033_a02e_4c08_9c47_57a5cd70acc2.slice/crio-714e7f19ea33eb516fcbfbe0f338d156e2ea2970993738b8f28f3d40eac4f135 WatchSource:0}: Error finding container 714e7f19ea33eb516fcbfbe0f338d156e2ea2970993738b8f28f3d40eac4f135: Status 404 returned error can't find the container with id 714e7f19ea33eb516fcbfbe0f338d156e2ea2970993738b8f28f3d40eac4f135
Jul 10 14:43:42.511284 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.511259    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:43:42.557675 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.557640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a908618e-c152-4b10-9f50-3037bfec9cf5-config-file\") pod \"limitador-limitador-54d6d988d5-2987p\" (UID: \"a908618e-c152-4b10-9f50-3037bfec9cf5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.557774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.557688    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsgdq\" (UniqueName: \"kubernetes.io/projected/a908618e-c152-4b10-9f50-3037bfec9cf5-kube-api-access-gsgdq\") pod \"limitador-limitador-54d6d988d5-2987p\" (UID: \"a908618e-c152-4b10-9f50-3037bfec9cf5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.658156 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.658091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a908618e-c152-4b10-9f50-3037bfec9cf5-config-file\") pod \"limitador-limitador-54d6d988d5-2987p\" (UID: \"a908618e-c152-4b10-9f50-3037bfec9cf5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.658156 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.658126    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsgdq\" (UniqueName: \"kubernetes.io/projected/a908618e-c152-4b10-9f50-3037bfec9cf5-kube-api-access-gsgdq\") pod \"limitador-limitador-54d6d988d5-2987p\" (UID: \"a908618e-c152-4b10-9f50-3037bfec9cf5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.658727 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.658709    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a908618e-c152-4b10-9f50-3037bfec9cf5-config-file\") pod \"limitador-limitador-54d6d988d5-2987p\" (UID: \"a908618e-c152-4b10-9f50-3037bfec9cf5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.667766 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.667745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsgdq\" (UniqueName: \"kubernetes.io/projected/a908618e-c152-4b10-9f50-3037bfec9cf5-kube-api-access-gsgdq\") pod \"limitador-limitador-54d6d988d5-2987p\" (UID: \"a908618e-c152-4b10-9f50-3037bfec9cf5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.777789 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.777761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:42.922813 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:42.922788    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:43:42.925019 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:43:42.924995    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda908618e_c152_4b10_9f50_3037bfec9cf5.slice/crio-30064f42d9fd0cc3b5e506f6a4e18e0b63c578aa57641f7a1714ae1c4d27924c WatchSource:0}: Error finding container 30064f42d9fd0cc3b5e506f6a4e18e0b63c578aa57641f7a1714ae1c4d27924c: Status 404 returned error can't find the container with id 30064f42d9fd0cc3b5e506f6a4e18e0b63c578aa57641f7a1714ae1c4d27924c
Jul 10 14:43:43.279315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:43.279134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5" event={"ID":"19031033-a02e-4c08-9c47-57a5cd70acc2","Type":"ContainerStarted","Data":"714e7f19ea33eb516fcbfbe0f338d156e2ea2970993738b8f28f3d40eac4f135"}
Jul 10 14:43:43.281616 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:43.281587    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p" event={"ID":"a908618e-c152-4b10-9f50-3037bfec9cf5","Type":"ContainerStarted","Data":"30064f42d9fd0cc3b5e506f6a4e18e0b63c578aa57641f7a1714ae1c4d27924c"}
Jul 10 14:43:43.317003 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:43.316972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:43.408332 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:43.408282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:46.297801 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:46.297768    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p" event={"ID":"a908618e-c152-4b10-9f50-3037bfec9cf5","Type":"ContainerStarted","Data":"2a98ea3d27ed385d2cb8edbaa81ac253071f960726c5c945bdf68323b000029d"}
Jul 10 14:43:46.298260 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:46.297919    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:46.299311 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:46.299286    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5" event={"ID":"19031033-a02e-4c08-9c47-57a5cd70acc2","Type":"ContainerStarted","Data":"81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd"}
Jul 10 14:43:46.299450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:46.299380    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:46.317404 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:46.317358    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p" podStartSLOduration=1.9152032380000001 podStartE2EDuration="4.317345454s" totalImagesPullingTime="2.402142216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:43:46.316218413 +0000 UTC m=+524.022539880" watchObservedRunningTime="2026-07-10 14:43:46.317345454 +0000 UTC m=+524.023666968"
Jul 10 14:43:46.334394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:46.334352    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5" podStartSLOduration=1.519640448 podStartE2EDuration="4.33433532s" totalImagesPullingTime="2.814694872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:43:46.333632235 +0000 UTC m=+524.039953726" watchObservedRunningTime="2026-07-10 14:43:46.33433532 +0000 UTC m=+524.040656790"
Jul 10 14:43:57.305492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:57.305420    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2987p"
Jul 10 14:43:57.305492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:57.305483    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:57.377492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:57.377463    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:57.377661 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:57.377626    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5" podUID="19031033-a02e-4c08-9c47-57a5cd70acc2" containerName="limitador" containerID="cri-o://81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd" gracePeriod=30
Jul 10 14:43:57.944243 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:57.943976    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:58.086438 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.086412    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19031033-a02e-4c08-9c47-57a5cd70acc2-config-file\") pod \"19031033-a02e-4c08-9c47-57a5cd70acc2\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") "
Jul 10 14:43:58.086626 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.086505    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp8mc\" (UniqueName: \"kubernetes.io/projected/19031033-a02e-4c08-9c47-57a5cd70acc2-kube-api-access-bp8mc\") pod \"19031033-a02e-4c08-9c47-57a5cd70acc2\" (UID: \"19031033-a02e-4c08-9c47-57a5cd70acc2\") "
Jul 10 14:43:58.086810 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.086789    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19031033-a02e-4c08-9c47-57a5cd70acc2-config-file" (OuterVolumeSpecName: "config-file") pod "19031033-a02e-4c08-9c47-57a5cd70acc2" (UID: "19031033-a02e-4c08-9c47-57a5cd70acc2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:43:58.088465 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.088428    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19031033-a02e-4c08-9c47-57a5cd70acc2-kube-api-access-bp8mc" (OuterVolumeSpecName: "kube-api-access-bp8mc") pod "19031033-a02e-4c08-9c47-57a5cd70acc2" (UID: "19031033-a02e-4c08-9c47-57a5cd70acc2"). InnerVolumeSpecName "kube-api-access-bp8mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:43:58.187629 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.187594    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp8mc\" (UniqueName: \"kubernetes.io/projected/19031033-a02e-4c08-9c47-57a5cd70acc2-kube-api-access-bp8mc\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:43:58.187629 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.187623    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19031033-a02e-4c08-9c47-57a5cd70acc2-config-file\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:43:58.357476 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.357393    2583 generic.go:356] "Generic (PLEG): container finished" podID="19031033-a02e-4c08-9c47-57a5cd70acc2" containerID="81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd" exitCode=0
Jul 10 14:43:58.357476 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.357443    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"
Jul 10 14:43:58.357476 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.357471    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5" event={"ID":"19031033-a02e-4c08-9c47-57a5cd70acc2","Type":"ContainerDied","Data":"81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd"}
Jul 10 14:43:58.357976 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.357542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjbh5" event={"ID":"19031033-a02e-4c08-9c47-57a5cd70acc2","Type":"ContainerDied","Data":"714e7f19ea33eb516fcbfbe0f338d156e2ea2970993738b8f28f3d40eac4f135"}
Jul 10 14:43:58.357976 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.357571    2583 scope.go:117] "RemoveContainer" containerID="81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd"
Jul 10 14:43:58.368975 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.368914    2583 scope.go:117] "RemoveContainer" containerID="81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd"
Jul 10 14:43:58.369455 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:43:58.369431    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd\": container with ID starting with 81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd not found: ID does not exist" containerID="81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd"
Jul 10 14:43:58.369676 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.369463    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd"} err="failed to get container status \"81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd\": rpc error: code = NotFound desc = could not find container \"81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd\": container with ID starting with 81d43ef299842d17518007fe3ba9c6e19625c68d85ffdcdff277b55d9690f2bd not found: ID does not exist"
Jul 10 14:43:58.379701 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.379674    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:58.383826 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.383807    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjbh5"]
Jul 10 14:43:58.846105 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:58.846076    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19031033-a02e-4c08-9c47-57a5cd70acc2" path="/var/lib/kubelet/pods/19031033-a02e-4c08-9c47-57a5cd70acc2/volumes"
Jul 10 14:43:59.691807 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:59.691726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:59.793072 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:43:59.793036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:00.168819 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.168785    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"]
Jul 10 14:44:00.169227 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.169210    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19031033-a02e-4c08-9c47-57a5cd70acc2" containerName="limitador"
Jul 10 14:44:00.169320 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.169231    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="19031033-a02e-4c08-9c47-57a5cd70acc2" containerName="limitador"
Jul 10 14:44:00.169377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.169363    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="19031033-a02e-4c08-9c47-57a5cd70acc2" containerName="limitador"
Jul 10 14:44:00.173822 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.173799    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.176543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.176502    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 14:44:00.176543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.176532    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 14:44:00.176691 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.176534    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 14:44:00.177355 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.177339    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v5ng7\""
Jul 10 14:44:00.181974 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.181953    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"]
Jul 10 14:44:00.223664 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.223632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:00.304566 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.304540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd7f7bf4-3555-445e-91ec-15413bbd5cdb-data\") pod \"postgres-69fb4594cf-tbp4z\" (UID: \"fd7f7bf4-3555-445e-91ec-15413bbd5cdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.304666 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.304579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzwrp\" (UniqueName: \"kubernetes.io/projected/fd7f7bf4-3555-445e-91ec-15413bbd5cdb-kube-api-access-hzwrp\") pod \"postgres-69fb4594cf-tbp4z\" (UID: \"fd7f7bf4-3555-445e-91ec-15413bbd5cdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.404946 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.404916    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd7f7bf4-3555-445e-91ec-15413bbd5cdb-data\") pod \"postgres-69fb4594cf-tbp4z\" (UID: \"fd7f7bf4-3555-445e-91ec-15413bbd5cdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.405052 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.404954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzwrp\" (UniqueName: \"kubernetes.io/projected/fd7f7bf4-3555-445e-91ec-15413bbd5cdb-kube-api-access-hzwrp\") pod \"postgres-69fb4594cf-tbp4z\" (UID: \"fd7f7bf4-3555-445e-91ec-15413bbd5cdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.405342 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.405324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd7f7bf4-3555-445e-91ec-15413bbd5cdb-data\") pod \"postgres-69fb4594cf-tbp4z\" (UID: \"fd7f7bf4-3555-445e-91ec-15413bbd5cdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.414158 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.414132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzwrp\" (UniqueName: \"kubernetes.io/projected/fd7f7bf4-3555-445e-91ec-15413bbd5cdb-kube-api-access-hzwrp\") pod \"postgres-69fb4594cf-tbp4z\" (UID: \"fd7f7bf4-3555-445e-91ec-15413bbd5cdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.484656 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.484606    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:00.612348 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:00.612322    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"]
Jul 10 14:44:00.614472 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:44:00.614447    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7f7bf4_3555_445e_91ec_15413bbd5cdb.slice/crio-1b6266c68984985b64e88bc88ba9df85b517aee1cc53a982c6156b3b0cfe696c WatchSource:0}: Error finding container 1b6266c68984985b64e88bc88ba9df85b517aee1cc53a982c6156b3b0cfe696c: Status 404 returned error can't find the container with id 1b6266c68984985b64e88bc88ba9df85b517aee1cc53a982c6156b3b0cfe696c
Jul 10 14:44:01.381344 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:01.381305    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z" event={"ID":"fd7f7bf4-3555-445e-91ec-15413bbd5cdb","Type":"ContainerStarted","Data":"1b6266c68984985b64e88bc88ba9df85b517aee1cc53a982c6156b3b0cfe696c"}
Jul 10 14:44:06.226562 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:06.226537    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 14:44:06.413131 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:06.413093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z" event={"ID":"fd7f7bf4-3555-445e-91ec-15413bbd5cdb","Type":"ContainerStarted","Data":"9b300e9813965cabc632a40f151acaf7fe9398ff8a1ba2f7ed0e997729aa39f6"}
Jul 10 14:44:06.413280 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:06.413183    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:06.434701 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:06.434650    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z" podStartSLOduration=0.826710607 podStartE2EDuration="6.434635558s" totalImagesPullingTime="5.607924951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:06.433020217 +0000 UTC m=+544.139341685" watchObservedRunningTime="2026-07-10 14:44:06.434635558 +0000 UTC m=+544.140957029"
Jul 10 14:44:12.447119 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:12.447088    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tbp4z"
Jul 10 14:44:12.772394 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:12.772320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:13.257806 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:13.257773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:19.764592 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.764556    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86db68b845-8xlrx"]
Jul 10 14:44:19.769450 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.769429    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:19.769855 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.769834    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:19.769931 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.769917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnc2j\" (UniqueName: \"kubernetes.io/projected/d04ba518-1add-426f-a407-c0a7f02cbbb1-kube-api-access-bnc2j\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:19.783293 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.783272    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 14:44:19.783751 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.783734    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cwn6x\""
Jul 10 14:44:19.784267 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.784252    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 14:44:19.797645 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.797627    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86db68b845-8xlrx"]
Jul 10 14:44:19.870436 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.870400    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnc2j\" (UniqueName: \"kubernetes.io/projected/d04ba518-1add-426f-a407-c0a7f02cbbb1-kube-api-access-bnc2j\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:19.870699 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.870464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:19.870699 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:44:19.870632    2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 10 14:44:19.870835 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:44:19.870711    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert podName:d04ba518-1add-426f-a407-c0a7f02cbbb1 nodeName:}" failed. No retries permitted until 2026-07-10 14:44:20.370690033 +0000 UTC m=+558.077011493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert") pod "maas-controller-86db68b845-8xlrx" (UID: "d04ba518-1add-426f-a407-c0a7f02cbbb1") : secret "maas-controller-webhook-cert" not found
Jul 10 14:44:19.883100 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.883072    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnc2j\" (UniqueName: \"kubernetes.io/projected/d04ba518-1add-426f-a407-c0a7f02cbbb1-kube-api-access-bnc2j\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:19.987427 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.987385    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b89cd5547-s99h9"]
Jul 10 14:44:19.989991 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:19.989977    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.003133 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.003110    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-s99h9"]
Jul 10 14:44:20.071893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.071833    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bq8l\" (UniqueName: \"kubernetes.io/projected/97abb3de-d084-4628-9e9a-6b5190f91fe6-kube-api-access-5bq8l\") pod \"maas-controller-7b89cd5547-s99h9\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") " pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.071893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.071872    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97abb3de-d084-4628-9e9a-6b5190f91fe6-webhook-cert\") pod \"maas-controller-7b89cd5547-s99h9\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") " pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.172807 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.172780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bq8l\" (UniqueName: \"kubernetes.io/projected/97abb3de-d084-4628-9e9a-6b5190f91fe6-kube-api-access-5bq8l\") pod \"maas-controller-7b89cd5547-s99h9\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") " pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.172908 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.172818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97abb3de-d084-4628-9e9a-6b5190f91fe6-webhook-cert\") pod \"maas-controller-7b89cd5547-s99h9\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") " pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.175134 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.175111    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97abb3de-d084-4628-9e9a-6b5190f91fe6-webhook-cert\") pod \"maas-controller-7b89cd5547-s99h9\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") " pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.182042 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.182020    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bq8l\" (UniqueName: \"kubernetes.io/projected/97abb3de-d084-4628-9e9a-6b5190f91fe6-kube-api-access-5bq8l\") pod \"maas-controller-7b89cd5547-s99h9\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") " pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.300535 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.300490    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:20.358158 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.358124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:20.374403 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.374372    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:20.376853 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.376837    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert\") pod \"maas-controller-86db68b845-8xlrx\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") " pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:20.379681 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.379664    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:20.425240 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.424451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:20.438171 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.438145    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-s99h9"]
Jul 10 14:44:20.441395 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:44:20.441349    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97abb3de_d084_4628_9e9a_6b5190f91fe6.slice/crio-fb2e3e25f9a44b3d02643b1e1d48e1694d0b7106beafe10db6b4b4d26c0c89f8 WatchSource:0}: Error finding container fb2e3e25f9a44b3d02643b1e1d48e1694d0b7106beafe10db6b4b4d26c0c89f8: Status 404 returned error can't find the container with id fb2e3e25f9a44b3d02643b1e1d48e1694d0b7106beafe10db6b4b4d26c0c89f8
Jul 10 14:44:20.487581 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.487540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b89cd5547-s99h9" event={"ID":"97abb3de-d084-4628-9e9a-6b5190f91fe6","Type":"ContainerStarted","Data":"fb2e3e25f9a44b3d02643b1e1d48e1694d0b7106beafe10db6b4b4d26c0c89f8"}
Jul 10 14:44:20.516968 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:20.516945    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86db68b845-8xlrx"]
Jul 10 14:44:20.519588 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:44:20.519559    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd04ba518_1add_426f_a407_c0a7f02cbbb1.slice/crio-16b49c9fe39f15bb826e66a933e60333636048f41f9144eb7f8d7d3af9616a2a WatchSource:0}: Error finding container 16b49c9fe39f15bb826e66a933e60333636048f41f9144eb7f8d7d3af9616a2a: Status 404 returned error can't find the container with id 16b49c9fe39f15bb826e66a933e60333636048f41f9144eb7f8d7d3af9616a2a
Jul 10 14:44:21.496892 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:21.496818    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86db68b845-8xlrx" event={"ID":"d04ba518-1add-426f-a407-c0a7f02cbbb1","Type":"ContainerStarted","Data":"16b49c9fe39f15bb826e66a933e60333636048f41f9144eb7f8d7d3af9616a2a"}
Jul 10 14:44:23.348433 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.348404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:23.441247 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.441217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:23.509105 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.509073    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86db68b845-8xlrx" event={"ID":"d04ba518-1add-426f-a407-c0a7f02cbbb1","Type":"ContainerStarted","Data":"d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564"}
Jul 10 14:44:23.509289 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.509245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:23.510603 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.510581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b89cd5547-s99h9" event={"ID":"97abb3de-d084-4628-9e9a-6b5190f91fe6","Type":"ContainerStarted","Data":"513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28"}
Jul 10 14:44:23.510729 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.510627    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:23.532186 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.532142    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86db68b845-8xlrx" podStartSLOduration=1.91917235 podStartE2EDuration="4.532129552s" totalImagesPullingTime="2.612957202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:23.53123107 +0000 UTC m=+561.237552538" watchObservedRunningTime="2026-07-10 14:44:23.532129552 +0000 UTC m=+561.238451020"
Jul 10 14:44:23.557202 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:23.557156    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b89cd5547-s99h9" podStartSLOduration=1.862059104 podStartE2EDuration="4.557145804s" totalImagesPullingTime="2.6950867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:23.556445406 +0000 UTC m=+561.262766877" watchObservedRunningTime="2026-07-10 14:44:23.557145804 +0000 UTC m=+561.263467272"
Jul 10 14:44:34.521914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.521883    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:44:34.522345 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.522051    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:34.589726 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.589702    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86db68b845-8xlrx"]
Jul 10 14:44:34.589889 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.589870    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86db68b845-8xlrx" podUID="d04ba518-1add-426f-a407-c0a7f02cbbb1" containerName="manager" containerID="cri-o://d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564" gracePeriod=30
Jul 10 14:44:34.840007 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.839983    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:34.896284 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.896251    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert\") pod \"d04ba518-1add-426f-a407-c0a7f02cbbb1\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") "
Jul 10 14:44:34.896457 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.896302    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnc2j\" (UniqueName: \"kubernetes.io/projected/d04ba518-1add-426f-a407-c0a7f02cbbb1-kube-api-access-bnc2j\") pod \"d04ba518-1add-426f-a407-c0a7f02cbbb1\" (UID: \"d04ba518-1add-426f-a407-c0a7f02cbbb1\") "
Jul 10 14:44:34.898427 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.898395    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d04ba518-1add-426f-a407-c0a7f02cbbb1" (UID: "d04ba518-1add-426f-a407-c0a7f02cbbb1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:44:34.898562 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.898446    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d04ba518-1add-426f-a407-c0a7f02cbbb1-kube-api-access-bnc2j" (OuterVolumeSpecName: "kube-api-access-bnc2j") pod "d04ba518-1add-426f-a407-c0a7f02cbbb1" (UID: "d04ba518-1add-426f-a407-c0a7f02cbbb1"). InnerVolumeSpecName "kube-api-access-bnc2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:44:34.997038 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.997004    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnc2j\" (UniqueName: \"kubernetes.io/projected/d04ba518-1add-426f-a407-c0a7f02cbbb1-kube-api-access-bnc2j\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:44:34.997038 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:34.997033    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d04ba518-1add-426f-a407-c0a7f02cbbb1-webhook-cert\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:44:35.480955 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.480924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:35.574708 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.574677    2583 generic.go:356] "Generic (PLEG): container finished" podID="d04ba518-1add-426f-a407-c0a7f02cbbb1" containerID="d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564" exitCode=0
Jul 10 14:44:35.575048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.574734    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86db68b845-8xlrx"
Jul 10 14:44:35.575048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.574759    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86db68b845-8xlrx" event={"ID":"d04ba518-1add-426f-a407-c0a7f02cbbb1","Type":"ContainerDied","Data":"d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564"}
Jul 10 14:44:35.575048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.574810    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86db68b845-8xlrx" event={"ID":"d04ba518-1add-426f-a407-c0a7f02cbbb1","Type":"ContainerDied","Data":"16b49c9fe39f15bb826e66a933e60333636048f41f9144eb7f8d7d3af9616a2a"}
Jul 10 14:44:35.575048 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.574832    2583 scope.go:117] "RemoveContainer" containerID="d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564"
Jul 10 14:44:35.583855 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.583833    2583 scope.go:117] "RemoveContainer" containerID="d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564"
Jul 10 14:44:35.584078 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:44:35.584058    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564\": container with ID starting with d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564 not found: ID does not exist" containerID="d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564"
Jul 10 14:44:35.584125 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.584086    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564"} err="failed to get container status \"d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564\": rpc error: code = NotFound desc = could not find container \"d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564\": container with ID starting with d6673c057ec074c06a4f617b95695ab9b9aeeb308550ce7fb360e09005b15564 not found: ID does not exist"
Jul 10 14:44:35.596904 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.596878    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86db68b845-8xlrx"]
Jul 10 14:44:35.601527 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:35.601498    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86db68b845-8xlrx"]
Jul 10 14:44:36.846429 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:36.846396    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d04ba518-1add-426f-a407-c0a7f02cbbb1" path="/var/lib/kubelet/pods/d04ba518-1add-426f-a407-c0a7f02cbbb1/volumes"
Jul 10 14:44:55.858703 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:55.858672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:56.329107 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.329077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:56.384845 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.384815    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"]
Jul 10 14:44:56.385212 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.385201    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04ba518-1add-426f-a407-c0a7f02cbbb1" containerName="manager"
Jul 10 14:44:56.385255 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.385214    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04ba518-1add-426f-a407-c0a7f02cbbb1" containerName="manager"
Jul 10 14:44:56.385289 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.385280    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d04ba518-1add-426f-a407-c0a7f02cbbb1" containerName="manager"
Jul 10 14:44:56.388198 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.388182    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.390814 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.390793    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 14:44:56.390919 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.390853    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8qgwq\""
Jul 10 14:44:56.397574 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.397542    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"]
Jul 10 14:44:56.477881 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.477850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa59778-99ff-4210-a681-91dd3b970a9e-maas-api-tls\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.478027 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.477897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa59778-99ff-4210-a681-91dd3b970a9e-openshift-service-ca\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.478027 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.477977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psbl6\" (UniqueName: \"kubernetes.io/projected/efa59778-99ff-4210-a681-91dd3b970a9e-kube-api-access-psbl6\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.558628 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.558589    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:56.578806 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.578779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa59778-99ff-4210-a681-91dd3b970a9e-maas-api-tls\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.578914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.578822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa59778-99ff-4210-a681-91dd3b970a9e-openshift-service-ca\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.578914 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.578867    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psbl6\" (UniqueName: \"kubernetes.io/projected/efa59778-99ff-4210-a681-91dd3b970a9e-kube-api-access-psbl6\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.579573 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.579504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa59778-99ff-4210-a681-91dd3b970a9e-openshift-service-ca\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.581174 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.581153    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa59778-99ff-4210-a681-91dd3b970a9e-maas-api-tls\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.586674 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.586652    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psbl6\" (UniqueName: \"kubernetes.io/projected/efa59778-99ff-4210-a681-91dd3b970a9e-kube-api-access-psbl6\") pod \"maas-api-5948dcb657-7q2rk\" (UID: \"efa59778-99ff-4210-a681-91dd3b970a9e\") " pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.698559 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.698525    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:56.809462 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.809424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:56.843019 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:44:56.842988    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa59778_99ff_4210_a681_91dd3b970a9e.slice/crio-9e5d5332e37e56c9f127c388032c3452238b91db254b87cafa10242385a3e2fc WatchSource:0}: Error finding container 9e5d5332e37e56c9f127c388032c3452238b91db254b87cafa10242385a3e2fc: Status 404 returned error can't find the container with id 9e5d5332e37e56c9f127c388032c3452238b91db254b87cafa10242385a3e2fc
Jul 10 14:44:56.847422 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.847392    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"]
Jul 10 14:44:56.905281 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.905247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:56.930978 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.930957    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gwjx9"]
Jul 10 14:44:56.935497 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.935481    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:56.938355 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.938339    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-z5wdk\""
Jul 10 14:44:56.938580 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.938560    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 14:44:56.948005 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:56.947985    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gwjx9"]
Jul 10 14:44:57.062566 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.062537    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:44:57.083458 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.083417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gwjx9\" (UID: \"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f\") " pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.083458 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.083467    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wk7z\" (UniqueName: \"kubernetes.io/projected/b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f-kube-api-access-8wk7z\") pod \"payload-pre-processing-7964457696-gwjx9\" (UID: \"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f\") " pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.183969 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.183931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gwjx9\" (UID: \"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f\") " pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.184145 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.183984    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wk7z\" (UniqueName: \"kubernetes.io/projected/b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f-kube-api-access-8wk7z\") pod \"payload-pre-processing-7964457696-gwjx9\" (UID: \"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f\") " pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.184654 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.184630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gwjx9\" (UID: \"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f\") " pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.192927 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.192905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wk7z\" (UniqueName: \"kubernetes.io/projected/b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f-kube-api-access-8wk7z\") pod \"payload-pre-processing-7964457696-gwjx9\" (UID: \"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f\") " pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.246733 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.246700    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:57.383310 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.383286    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gwjx9"]
Jul 10 14:44:57.385552 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:44:57.385505    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98c6a9a_9a25_48eb_a5a8_33e0ea580c2f.slice/crio-8a5c021c4d58651a10487e0e5665ebf8d2ef7a109c16f598600b10b127512093 WatchSource:0}: Error finding container 8a5c021c4d58651a10487e0e5665ebf8d2ef7a109c16f598600b10b127512093: Status 404 returned error can't find the container with id 8a5c021c4d58651a10487e0e5665ebf8d2ef7a109c16f598600b10b127512093
Jul 10 14:44:57.686591 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.686552    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9" event={"ID":"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f","Type":"ContainerStarted","Data":"8a5c021c4d58651a10487e0e5665ebf8d2ef7a109c16f598600b10b127512093"}
Jul 10 14:44:57.687933 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:57.687902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk" event={"ID":"efa59778-99ff-4210-a681-91dd3b970a9e","Type":"ContainerStarted","Data":"9e5d5332e37e56c9f127c388032c3452238b91db254b87cafa10242385a3e2fc"}
Jul 10 14:44:59.700174 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:59.700139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk" event={"ID":"efa59778-99ff-4210-a681-91dd3b970a9e","Type":"ContainerStarted","Data":"7012309a4dac57e893949f425a2670ae67b1e8f69a86fa0c8732d7cc22e02993"}
Jul 10 14:44:59.700628 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:59.700207    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:44:59.701655 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:59.701632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9" event={"ID":"b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f","Type":"ContainerStarted","Data":"037d65a7e8bc881134fe4c317a32e171f70f991de72eccc551ee3ad2863a6ad1"}
Jul 10 14:44:59.701781 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:59.701757    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:44:59.722919 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:59.722881    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk" podStartSLOduration=1.265850176 podStartE2EDuration="3.722867012s" totalImagesPullingTime="2.457016836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:59.720253242 +0000 UTC m=+597.426574701" watchObservedRunningTime="2026-07-10 14:44:59.722867012 +0000 UTC m=+597.429188502"
Jul 10 14:44:59.740393 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:44:59.740345    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9" podStartSLOduration=1.823532318 podStartE2EDuration="3.740324977s" totalImagesPullingTime="1.916792659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:59.738505116 +0000 UTC m=+597.444826586" watchObservedRunningTime="2026-07-10 14:44:59.740324977 +0000 UTC m=+597.446646446"
Jul 10 14:45:00.153991 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.153902    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"]
Jul 10 14:45:00.158861 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.158834    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"
Jul 10 14:45:00.175300 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.175280    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"]
Jul 10 14:45:00.177187 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.177171    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gj49t\""
Jul 10 14:45:00.180933 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.180913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"
Jul 10 14:45:00.355561 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.345864    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"]
Jul 10 14:45:00.708164 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:00.708127    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerStarted","Data":"56988a38af0aa756af97b2031ffed6345b319494b424df0caa80fb8e2c502a20"}
Jul 10 14:45:02.976343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:02.976315    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:45:02.976959 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:02.976934    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:45:03.727896 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:03.727853    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerStarted","Data":"66031ddfe324c652df7acd3f28582385e007000b2167429a0da8c0799b3d3c95"}
Jul 10 14:45:03.745929 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:03.745885    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" podStartSLOduration=1.104728313 podStartE2EDuration="3.745870281s" totalImagesPullingTime="2.641141968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:03.744827827 +0000 UTC m=+601.451149297" watchObservedRunningTime="2026-07-10 14:45:03.745870281 +0000 UTC m=+601.452191728"
Jul 10 14:45:04.736179 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.736149    2583 generic.go:356] "Generic (PLEG): container finished" podID="e0f16b21-14c2-4081-b5a8-6699f5840a7b" containerID="60a0bbfec1a6fe975a72f01e53e94e015336714e66987020586372ef39e58012" exitCode=0
Jul 10 14:45:04.736640 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.736230    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" event={"ID":"e0f16b21-14c2-4081-b5a8-6699f5840a7b","Type":"ContainerDied","Data":"60a0bbfec1a6fe975a72f01e53e94e015336714e66987020586372ef39e58012"}
Jul 10 14:45:04.736721 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.736697    2583 scope.go:117] "RemoveContainer" containerID="60a0bbfec1a6fe975a72f01e53e94e015336714e66987020586372ef39e58012"
Jul 10 14:45:04.737789 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.737764    2583 generic.go:356] "Generic (PLEG): container finished" podID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerID="66031ddfe324c652df7acd3f28582385e007000b2167429a0da8c0799b3d3c95" exitCode=7
Jul 10 14:45:04.737893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.737772    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:45:04.737893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.737790    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerDied","Data":"66031ddfe324c652df7acd3f28582385e007000b2167429a0da8c0799b3d3c95"}
Jul 10 14:45:04.737893 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.737860    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:45:04.738011 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.737905    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 14:45:04.738011 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:04.737971    2583 scope.go:117] "RemoveContainer" containerID="66031ddfe324c652df7acd3f28582385e007000b2167429a0da8c0799b3d3c95"
Jul 10 14:45:05.713922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.713891    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5948dcb657-7q2rk"
Jul 10 14:45:05.759125 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.758128    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" event={"ID":"e0f16b21-14c2-4081-b5a8-6699f5840a7b","Type":"ContainerStarted","Data":"493251916c8464794b3333da1f81414e4056717500de292bcdbb53112318b51b"}
Jul 10 14:45:05.763119 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.762012    2583 generic.go:356] "Generic (PLEG): container finished" podID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerID="9cdbeb9a4fdf297c59a1c5e8027b6649f825c73f9a20b1f7e6467cfcd4588998" exitCode=7
Jul 10 14:45:05.763119 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.762104    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerDied","Data":"9cdbeb9a4fdf297c59a1c5e8027b6649f825c73f9a20b1f7e6467cfcd4588998"}
Jul 10 14:45:05.763119 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.762154    2583 scope.go:117] "RemoveContainer" containerID="66031ddfe324c652df7acd3f28582385e007000b2167429a0da8c0799b3d3c95"
Jul 10 14:45:05.763119 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.762564    2583 scope.go:117] "RemoveContainer" containerID="9cdbeb9a4fdf297c59a1c5e8027b6649f825c73f9a20b1f7e6467cfcd4588998"
Jul 10 14:45:05.763119 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:45:05.762873    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29728245-7nn7q_odh-ai-gateway-infra(05c1c436-4bf3-4a4b-a996-4c2d04149f6a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a"
Jul 10 14:45:05.791672 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.791644    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:45:05.791859 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.791839    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pzt5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 10 14:45:05.791947 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:05.791888    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" podUID="e0f16b21-14c2-4081-b5a8-6699f5840a7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 10 14:45:06.718286 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:06.718196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:06.769302 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:06.769275    2583 scope.go:117] "RemoveContainer" containerID="9cdbeb9a4fdf297c59a1c5e8027b6649f825c73f9a20b1f7e6467cfcd4588998"
Jul 10 14:45:06.769704 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:45:06.769493    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29728245-7nn7q_odh-ai-gateway-infra(05c1c436-4bf3-4a4b-a996-4c2d04149f6a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a"
Jul 10 14:45:06.791804 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:06.791776    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pzt5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 10 14:45:06.791935 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:06.791821    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" podUID="e0f16b21-14c2-4081-b5a8-6699f5840a7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 10 14:45:07.792133 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:07.792102    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pzt5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 10 14:45:07.792600 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:07.792154    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" podUID="e0f16b21-14c2-4081-b5a8-6699f5840a7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 10 14:45:08.233125 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:08.232992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:08.791824 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:08.791794    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-pzt5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 10 14:45:08.791990 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:08.791843    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b" podUID="e0f16b21-14c2-4081-b5a8-6699f5840a7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 10 14:45:09.834938 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:09.834909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:45:09.835312 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:09.835187    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:45:09.836115 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:09.836092    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-pzt5b"
Jul 10 14:45:10.711188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:10.711156    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-gwjx9"
Jul 10 14:45:18.644233 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:18.644196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:18.743644 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:18.743614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:18.840415 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:18.840385    2583 scope.go:117] "RemoveContainer" containerID="9cdbeb9a4fdf297c59a1c5e8027b6649f825c73f9a20b1f7e6467cfcd4588998"
Jul 10 14:45:19.834441 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:19.834406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerStarted","Data":"b9a39f381f233385927ac0533736cacfcc677175c29600d1d25586d5e346ecd3"}
Jul 10 14:45:20.863613 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:20.863580    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"]
Jul 10 14:45:20.864030 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:20.863741    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup" containerID="cri-o://b9a39f381f233385927ac0533736cacfcc677175c29600d1d25586d5e346ecd3" gracePeriod=30
Jul 10 14:45:30.011847 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.011771    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"]
Jul 10 14:45:30.016116 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.016096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.019841 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.019820    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 14:45:30.019954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.019826    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wbj7h\""
Jul 10 14:45:30.019954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.019883    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 14:45:30.019954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.019883    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 10 14:45:30.027913 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.027893    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"]
Jul 10 14:45:30.040909 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.040876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:30.085334 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.085306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.085456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.085342    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.085456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.085360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2npz\" (UniqueName: \"kubernetes.io/projected/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-kube-api-access-s2npz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.085456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.085411    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.085615 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.085510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.085615 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.085596    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187057 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187019    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187109    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2npz\" (UniqueName: \"kubernetes.io/projected/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-kube-api-access-s2npz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187217    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187474 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187577 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187498    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187633 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187578    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.187810 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.187790    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.189323 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.189301    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.189578 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.189562    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.195073 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.195050    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2npz\" (UniqueName: \"kubernetes.io/projected/e50010f1-6ae8-4cc6-aeaa-6e010c30923b-kube-api-access-s2npz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj\" (UID: \"e50010f1-6ae8-4cc6-aeaa-6e010c30923b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.326770 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.326688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:30.465879 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.465848    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:45:30.470341 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.470321    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"]
Jul 10 14:45:30.473129 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:45:30.473107    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50010f1_6ae8_4cc6_aeaa_6e010c30923b.slice/crio-ab30d41186b9d70998495ff44b626463566e871c89ee06dd5878b646c6c3e9ec WatchSource:0}: Error finding container ab30d41186b9d70998495ff44b626463566e871c89ee06dd5878b646c6c3e9ec: Status 404 returned error can't find the container with id ab30d41186b9d70998495ff44b626463566e871c89ee06dd5878b646c6c3e9ec
Jul 10 14:45:30.474810 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.474793    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:45:30.892894 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:30.892853    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj" event={"ID":"e50010f1-6ae8-4cc6-aeaa-6e010c30923b","Type":"ContainerStarted","Data":"ab30d41186b9d70998495ff44b626463566e871c89ee06dd5878b646c6c3e9ec"}
Jul 10 14:45:31.899552 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:31.899443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj" event={"ID":"e50010f1-6ae8-4cc6-aeaa-6e010c30923b","Type":"ContainerStarted","Data":"d2cd5de570a28c6e6bac2dc11cedfa4a3bc6a71bf65111d1272386976a4194a5"}
Jul 10 14:45:31.924214 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:31.924168    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj" podStartSLOduration=1.800558082 podStartE2EDuration="2.924153269s" totalImagesPullingTime="1.123595187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:31.922653899 +0000 UTC m=+629.628975378" watchObservedRunningTime="2026-07-10 14:45:31.924153269 +0000 UTC m=+629.630474739"
Jul 10 14:45:40.327583 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:40.327548    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:40.328054 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:40.327612    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:40.339877 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:40.339846    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:40.958790 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:40.958762    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj"
Jul 10 14:45:48.449752 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:48.449714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:49.166336 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:49.166302    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:45:50.999130 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:50.999100    2583 generic.go:356] "Generic (PLEG): container finished" podID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerID="b9a39f381f233385927ac0533736cacfcc677175c29600d1d25586d5e346ecd3" exitCode=137
Jul 10 14:45:50.999480 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:50.999169    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerDied","Data":"b9a39f381f233385927ac0533736cacfcc677175c29600d1d25586d5e346ecd3"}
Jul 10 14:45:50.999480 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:50.999229    2583 scope.go:117] "RemoveContainer" containerID="9cdbeb9a4fdf297c59a1c5e8027b6649f825c73f9a20b1f7e6467cfcd4588998"
Jul 10 14:45:51.069537 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:51.069490    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"
Jul 10 14:45:51.337996 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:51.337968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:52.006390 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:52.006355    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q" event={"ID":"05c1c436-4bf3-4a4b-a996-4c2d04149f6a","Type":"ContainerDied","Data":"56988a38af0aa756af97b2031ffed6345b319494b424df0caa80fb8e2c502a20"}
Jul 10 14:45:52.006832 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:52.006410    2583 scope.go:117] "RemoveContainer" containerID="b9a39f381f233385927ac0533736cacfcc677175c29600d1d25586d5e346ecd3"
Jul 10 14:45:52.006832 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:52.006371    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"
Jul 10 14:45:52.027038 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:52.027011    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"]
Jul 10 14:45:52.031556 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:52.031509    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728245-7nn7q"]
Jul 10 14:45:52.846649 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:45:52.846610    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" path="/var/lib/kubelet/pods/05c1c436-4bf3-4a4b-a996-4c2d04149f6a/volumes"
Jul 10 14:46:00.201233 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201199    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"]
Jul 10 14:46:00.201683 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201669    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201737 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201684    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201737 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201696    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201737 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201705    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201737 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201716    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201737 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201722    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201898 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201785    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.201898 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.201798    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:46:00.204965 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.204947    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.207702 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.207685    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 14:46:00.218491 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.218468    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"]
Jul 10 14:46:00.239326 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.239292    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:00.358327 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.358289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.358492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.358338    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.358492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.358360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.358492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.358422    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbebd686-8880-41c3-a764-f1ef1beea8e9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.358492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.358466    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.358492 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.358490    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxj7k\" (UniqueName: \"kubernetes.io/projected/bbebd686-8880-41c3-a764-f1ef1beea8e9-kube-api-access-mxj7k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.459874 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.459787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.459874 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.459834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxj7k\" (UniqueName: \"kubernetes.io/projected/bbebd686-8880-41c3-a764-f1ef1beea8e9-kube-api-access-mxj7k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460101 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.459888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460101 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.459921    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460101 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.460070    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460258 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.460124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbebd686-8880-41c3-a764-f1ef1beea8e9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460307 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.460262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460359 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.460337    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.460475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.460455    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.462049 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.462015    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbebd686-8880-41c3-a764-f1ef1beea8e9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.462430 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.462409    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbebd686-8880-41c3-a764-f1ef1beea8e9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.468208 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.468188    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxj7k\" (UniqueName: \"kubernetes.io/projected/bbebd686-8880-41c3-a764-f1ef1beea8e9-kube-api-access-mxj7k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-84xck\" (UID: \"bbebd686-8880-41c3-a764-f1ef1beea8e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.514924 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.514896    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:00.654269 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:00.654242    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"]
Jul 10 14:46:00.658000 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:46:00.657942    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbebd686_8880_41c3_a764_f1ef1beea8e9.slice/crio-bf186c67ffaaec1ca8c3fb34051df40249ddebdfb73da84cca6df8e7f7051453 WatchSource:0}: Error finding container bf186c67ffaaec1ca8c3fb34051df40249ddebdfb73da84cca6df8e7f7051453: Status 404 returned error can't find the container with id bf186c67ffaaec1ca8c3fb34051df40249ddebdfb73da84cca6df8e7f7051453
Jul 10 14:46:01.053769 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:01.053727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck" event={"ID":"bbebd686-8880-41c3-a764-f1ef1beea8e9","Type":"ContainerStarted","Data":"a1ef23f8289dbeef3766884cd67f511e179673ee028b31d44f33a1540bb076dc"}
Jul 10 14:46:01.053925 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:01.053788    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck" event={"ID":"bbebd686-8880-41c3-a764-f1ef1beea8e9","Type":"ContainerStarted","Data":"bf186c67ffaaec1ca8c3fb34051df40249ddebdfb73da84cca6df8e7f7051453"}
Jul 10 14:46:01.076989 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:01.076944    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck" podStartSLOduration=0.868901222 podStartE2EDuration="1.076929488s" totalImagesPullingTime="208.028266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:01.076099376 +0000 UTC m=+658.782420845" watchObservedRunningTime="2026-07-10 14:46:01.076929488 +0000 UTC m=+658.783250957"
Jul 10 14:46:01.273995 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:01.273959    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:46:10.515543 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:10.515484    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:10.516008 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:10.515581    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:10.528253 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:10.528220    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:11.116076 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:11.116048    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-84xck"
Jul 10 14:46:13.166327 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:13.166297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:13.225600 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:13.225547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:17.549377 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:17.549338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:19.168989 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:19.168956    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:46:24.434952 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:24.434909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:25.673486 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:25.673452    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:46:31.224412 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:31.224378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:34.077532 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:34.077495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:46:46.198441 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:46.198405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:57.569070 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:46:57.568992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:24.736769 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:47:24.736735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:30.833306 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:47:30.833270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:30.939471 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:47:30.939433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:31.992065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:47:31.992033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:32.108239 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:47:32.108206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:59.767087 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:47:59.767047    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:48:09.866318 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:09.866274    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:48:18.633841 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:18.633802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:20.164554 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:20.164501    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:48:30.073980 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:30.073901    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:48:40.492928 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:40.492888    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:48:42.815455 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:42.815420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:42.909254 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:42.909225    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-s99h9"]
Jul 10 14:48:42.909426 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:42.909406    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b89cd5547-s99h9" podUID="97abb3de-d084-4628-9e9a-6b5190f91fe6" containerName="manager" containerID="cri-o://513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28" gracePeriod=30
Jul 10 14:48:43.161477 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.161452    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:48:43.340099 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.340013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bq8l\" (UniqueName: \"kubernetes.io/projected/97abb3de-d084-4628-9e9a-6b5190f91fe6-kube-api-access-5bq8l\") pod \"97abb3de-d084-4628-9e9a-6b5190f91fe6\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") "
Jul 10 14:48:43.340271 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.340100    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97abb3de-d084-4628-9e9a-6b5190f91fe6-webhook-cert\") pod \"97abb3de-d084-4628-9e9a-6b5190f91fe6\" (UID: \"97abb3de-d084-4628-9e9a-6b5190f91fe6\") "
Jul 10 14:48:43.342065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.342039    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97abb3de-d084-4628-9e9a-6b5190f91fe6-kube-api-access-5bq8l" (OuterVolumeSpecName: "kube-api-access-5bq8l") pod "97abb3de-d084-4628-9e9a-6b5190f91fe6" (UID: "97abb3de-d084-4628-9e9a-6b5190f91fe6"). InnerVolumeSpecName "kube-api-access-5bq8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:48:43.342065 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.342045    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97abb3de-d084-4628-9e9a-6b5190f91fe6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "97abb3de-d084-4628-9e9a-6b5190f91fe6" (UID: "97abb3de-d084-4628-9e9a-6b5190f91fe6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:48:43.441709 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.441678    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bq8l\" (UniqueName: \"kubernetes.io/projected/97abb3de-d084-4628-9e9a-6b5190f91fe6-kube-api-access-5bq8l\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:48:43.441709 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.441704    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97abb3de-d084-4628-9e9a-6b5190f91fe6-webhook-cert\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 14:48:43.905261 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.905229    2583 generic.go:356] "Generic (PLEG): container finished" podID="97abb3de-d084-4628-9e9a-6b5190f91fe6" containerID="513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28" exitCode=0
Jul 10 14:48:43.905739 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.905276    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b89cd5547-s99h9"
Jul 10 14:48:43.905739 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.905310    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b89cd5547-s99h9" event={"ID":"97abb3de-d084-4628-9e9a-6b5190f91fe6","Type":"ContainerDied","Data":"513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28"}
Jul 10 14:48:43.905739 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.905370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b89cd5547-s99h9" event={"ID":"97abb3de-d084-4628-9e9a-6b5190f91fe6","Type":"ContainerDied","Data":"fb2e3e25f9a44b3d02643b1e1d48e1694d0b7106beafe10db6b4b4d26c0c89f8"}
Jul 10 14:48:43.905739 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.905408    2583 scope.go:117] "RemoveContainer" containerID="513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28"
Jul 10 14:48:43.915584 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.915447    2583 scope.go:117] "RemoveContainer" containerID="513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28"
Jul 10 14:48:43.915916 ip-10-0-136-210 kubenswrapper[2583]: E0710 14:48:43.915885    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28\": container with ID starting with 513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28 not found: ID does not exist" containerID="513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28"
Jul 10 14:48:43.916025 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.915924    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28"} err="failed to get container status \"513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28\": rpc error: code = NotFound desc = could not find container \"513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28\": container with ID starting with 513186b33037872adda025123cf80087cae0212eff59a1eee89e166d35742a28 not found: ID does not exist"
Jul 10 14:48:43.938884 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.938855    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-s99h9"]
Jul 10 14:48:43.940895 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:43.940875    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-s99h9"]
Jul 10 14:48:44.665120 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.665084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:44.763862 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.763828    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b89cd5547-8cpt7"]
Jul 10 14:48:44.764320 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.764307    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97abb3de-d084-4628-9e9a-6b5190f91fe6" containerName="manager"
Jul 10 14:48:44.764370 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.764322    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="97abb3de-d084-4628-9e9a-6b5190f91fe6" containerName="manager"
Jul 10 14:48:44.764406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.764388    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="97abb3de-d084-4628-9e9a-6b5190f91fe6" containerName="manager"
Jul 10 14:48:44.764406 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.764397    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c1c436-4bf3-4a4b-a996-4c2d04149f6a" containerName="cleanup"
Jul 10 14:48:44.767263 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.767245    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:44.772109 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.772084    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cwn6x\""
Jul 10 14:48:44.772247 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.772088    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 14:48:44.772247 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.772134    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 14:48:44.784315 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.784293    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-8cpt7"]
Jul 10 14:48:44.846475 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.846438    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97abb3de-d084-4628-9e9a-6b5190f91fe6" path="/var/lib/kubelet/pods/97abb3de-d084-4628-9e9a-6b5190f91fe6/volumes"
Jul 10 14:48:44.953730 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.953663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrlld\" (UniqueName: \"kubernetes.io/projected/cd521ab1-8096-47e5-91f9-782408cf0dc2-kube-api-access-rrlld\") pod \"maas-controller-7b89cd5547-8cpt7\" (UID: \"cd521ab1-8096-47e5-91f9-782408cf0dc2\") " pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:44.954151 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:44.953882    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd521ab1-8096-47e5-91f9-782408cf0dc2-webhook-cert\") pod \"maas-controller-7b89cd5547-8cpt7\" (UID: \"cd521ab1-8096-47e5-91f9-782408cf0dc2\") " pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.054821 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.054791    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrlld\" (UniqueName: \"kubernetes.io/projected/cd521ab1-8096-47e5-91f9-782408cf0dc2-kube-api-access-rrlld\") pod \"maas-controller-7b89cd5547-8cpt7\" (UID: \"cd521ab1-8096-47e5-91f9-782408cf0dc2\") " pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.054977 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.054855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd521ab1-8096-47e5-91f9-782408cf0dc2-webhook-cert\") pod \"maas-controller-7b89cd5547-8cpt7\" (UID: \"cd521ab1-8096-47e5-91f9-782408cf0dc2\") " pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.057188 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.057169    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd521ab1-8096-47e5-91f9-782408cf0dc2-webhook-cert\") pod \"maas-controller-7b89cd5547-8cpt7\" (UID: \"cd521ab1-8096-47e5-91f9-782408cf0dc2\") " pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.070527 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.070489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrlld\" (UniqueName: \"kubernetes.io/projected/cd521ab1-8096-47e5-91f9-782408cf0dc2-kube-api-access-rrlld\") pod \"maas-controller-7b89cd5547-8cpt7\" (UID: \"cd521ab1-8096-47e5-91f9-782408cf0dc2\") " pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.077145 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.077125    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.221036 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.220966    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b89cd5547-8cpt7"]
Jul 10 14:48:45.224917 ip-10-0-136-210 kubenswrapper[2583]: W0710 14:48:45.224889    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd521ab1_8096_47e5_91f9_782408cf0dc2.slice/crio-84d97ab09d2249fad9176b125dca41eb1c01dafc1942ed63ab45c4dbab826d42 WatchSource:0}: Error finding container 84d97ab09d2249fad9176b125dca41eb1c01dafc1942ed63ab45c4dbab826d42: Status 404 returned error can't find the container with id 84d97ab09d2249fad9176b125dca41eb1c01dafc1942ed63ab45c4dbab826d42
Jul 10 14:48:45.918727 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.918686    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b89cd5547-8cpt7" event={"ID":"cd521ab1-8096-47e5-91f9-782408cf0dc2","Type":"ContainerStarted","Data":"1f90f40a1cf27c501593b47c07608f17ee1baa22a852871a29f31483fc2a1fcf"}
Jul 10 14:48:45.918896 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.918754    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:48:45.918896 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.918781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b89cd5547-8cpt7" event={"ID":"cd521ab1-8096-47e5-91f9-782408cf0dc2","Type":"ContainerStarted","Data":"84d97ab09d2249fad9176b125dca41eb1c01dafc1942ed63ab45c4dbab826d42"}
Jul 10 14:48:45.963465 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:45.963419    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b89cd5547-8cpt7" podStartSLOduration=1.5449698509999998 podStartE2EDuration="1.96340447s" totalImagesPullingTime="418.434619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:48:45.961298633 +0000 UTC m=+823.667620117" watchObservedRunningTime="2026-07-10 14:48:45.96340447 +0000 UTC m=+823.669725939"
Jul 10 14:48:56.928774 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:48:56.928747    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b89cd5547-8cpt7"
Jul 10 14:49:08.171329 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:08.171296    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:49:18.759230 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:18.759175    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:49:25.196456 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:25.196419    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:49:25.240858 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:25.240807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:25.448253 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:25.448134    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:25.564956 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:25.564908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:32.105162 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:32.105107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:32.211324 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:32.211291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:32.332426 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:32.332391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:32.503428 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:32.503391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:36.590711 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:36.590663    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:49:39.482642 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:39.482610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:39.599541 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:39.599475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:49:56.185762 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:49:56.185685    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:03.009330 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:03.009298    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:50:03.010463 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:03.010439    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:50:03.168539 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:03.168491    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:13.666381 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:13.666340    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:19.072157 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:19.072113    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:30.570926 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:30.570888    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:39.968371 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:39.968340    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:50.661626 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:50.661592    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:50:58.175228 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:50:58.175195    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:51:05.863922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:51:05.863886    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:51:13.558861 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:51:13.558826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:51:26.372931 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:51:26.372848    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:51:35.338621 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:51:35.338591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:51:37.353149 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:51:37.353117    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:52:01.370167 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:52:01.370130    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:52:13.096441 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:52:13.096406    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:52:26.769227 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:52:26.769188    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:52:37.861093 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:52:37.861055    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:52:46.770214 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:52:46.770180    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:52:55.663831 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:52:55.663755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:53:04.580255 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:53:04.580220    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:54:56.067546 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:54:56.067501    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:54:56.768470 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:54:56.768438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:55:03.045556 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:03.045503    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:55:03.048849 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:03.048822    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 14:55:04.463932 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:04.463895    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:55:17.182715 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:17.182674    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:55:29.279395 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:29.279360    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:55:40.765343 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:40.765307    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:55:49.874701 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:49.874667    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:55:59.560206 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:55:59.560112    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:56:09.784418 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:56:09.784367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:56:10.063351 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:56:10.063262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:56:21.470484 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:56:21.470438    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:56:29.007771 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:56:29.007736    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:56:37.520049 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:56:37.520014    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:56:47.575732 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:56:47.575694    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:57:05.547473 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:57:05.547439    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:57:09.858365 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:57:09.858324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:57:14.072922 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:57:14.072881    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:57:25.169900 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:57:25.169817    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:57:37.267224 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:57:37.267184    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:57:53.770586 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:57:53.770503    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:58:05.070128 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:58:05.070090    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:58:14.160643 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:58:14.160608    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:58:24.395547 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:58:24.391995    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:58:33.159662 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:58:33.159629    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:58:42.111226 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:58:42.111193    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:58:55.762768 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:58:55.762675    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:59:04.367295 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:04.367256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:59:13.659770 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:13.659731    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:59:24.458954 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:24.458918    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:59:37.774564 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:37.774500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:59:46.183215 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:46.183178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 14:59:58.395906 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:58.395864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 14:59:58.782220 ip-10-0-136-210 kubenswrapper[2583]: I0710 14:59:58.782173    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:00:00.145384 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.145346    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"]
Jul 10 15:00:00.149312 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.149294    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"
Jul 10 15:00:00.152876 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.152854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gj49t\""
Jul 10 15:00:00.159350 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.159329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"
Jul 10 15:00:00.171207 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.171174    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"]
Jul 10 15:00:00.298524 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.298488    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"]
Jul 10 15:00:00.300909 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:00:00.300881    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40040c28_a114_46c2_af35_bc1632d5bebe.slice/crio-c02b137e3446d9e5f21edde6c1ae7489da6dea1243a0eeb6d57121e3778adf46 WatchSource:0}: Error finding container c02b137e3446d9e5f21edde6c1ae7489da6dea1243a0eeb6d57121e3778adf46: Status 404 returned error can't find the container with id c02b137e3446d9e5f21edde6c1ae7489da6dea1243a0eeb6d57121e3778adf46
Jul 10 15:00:00.382106 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:00.382070    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz" event={"ID":"40040c28-a114-46c2-af35-bc1632d5bebe","Type":"ContainerStarted","Data":"c02b137e3446d9e5f21edde6c1ae7489da6dea1243a0eeb6d57121e3778adf46"}
Jul 10 15:00:01.390410 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:01.390375    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz" event={"ID":"40040c28-a114-46c2-af35-bc1632d5bebe","Type":"ContainerStarted","Data":"f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3"}
Jul 10 15:00:01.412974 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:01.412919    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz" podStartSLOduration=1.412903588 podStartE2EDuration="1.412903588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:00:01.410487197 +0000 UTC m=+1499.116808667" watchObservedRunningTime="2026-07-10 15:00:01.412903588 +0000 UTC m=+1499.119225035"
Jul 10 15:00:03.082399 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:03.082369    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:00:03.085927 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:03.085904    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:00:09.674428 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:09.674387    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:00:20.475004 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:20.474967    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:00:22.007653 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:22.007611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:27.364939 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:27.364850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:31.672938 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:31.672898    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:00:35.628619 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:35.628587    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:40.749975 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:40.749943    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:00:40.790038 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:40.789997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:43.506136 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:43.506080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:43.811315 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:43.811224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:43.987257 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:43.987221    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:49.789387 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:49.789346    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:00:50.731241 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:50.731204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:50.806326 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:50.806289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:50.988357 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:50.988279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:51.089867 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:51.089828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:52.178843 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.178803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:52.265709 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.265678    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"]
Jul 10 15:00:52.269219 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.269197    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.272545 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.272526    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4a6b0771-serving-cert\""
Jul 10 15:00:52.284598 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.284576    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"]
Jul 10 15:00:52.303073 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.303047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:52.340098 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.340071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59r6d\" (UniqueName: \"kubernetes.io/projected/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-kube-api-access-59r6d\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.340243 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.340133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-openshift-service-ca\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.340243 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.340222    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-maas-api-tls\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.403157 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.403119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:52.441439 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.441347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-openshift-service-ca\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.441439 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.441414    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-maas-api-tls\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.441690 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.441504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59r6d\" (UniqueName: \"kubernetes.io/projected/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-kube-api-access-59r6d\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.442295 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.442218    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-openshift-service-ca\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.444562 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.444533    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-maas-api-tls\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.451411 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.451387    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59r6d\" (UniqueName: \"kubernetes.io/projected/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-kube-api-access-59r6d\") pod \"maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.579801 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.579747    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:52.735272 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.735239    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"]
Jul 10 15:00:52.737174 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:00:52.737136    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035bf7e6_f296_4dba_aec7_93ca4bad6ea8.slice/crio-a39abc466ac332455b19a87b9a9b569f7b105e491d5e7521ab42c4c26a4f47f7 WatchSource:0}: Error finding container a39abc466ac332455b19a87b9a9b569f7b105e491d5e7521ab42c4c26a4f47f7: Status 404 returned error can't find the container with id a39abc466ac332455b19a87b9a9b569f7b105e491d5e7521ab42c4c26a4f47f7
Jul 10 15:00:52.738626 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.738609    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 15:00:52.851070 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:52.851036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:53.200744 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:53.200715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:53.662879 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:53.662845    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6" event={"ID":"035bf7e6-f296-4dba-aec7-93ca4bad6ea8","Type":"ContainerStarted","Data":"fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46"}
Jul 10 15:00:53.663071 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:53.662909    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6" event={"ID":"035bf7e6-f296-4dba-aec7-93ca4bad6ea8","Type":"ContainerStarted","Data":"a39abc466ac332455b19a87b9a9b569f7b105e491d5e7521ab42c4c26a4f47f7"}
Jul 10 15:00:53.663071 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:53.662938    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:53.682885 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:53.682838    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6" podStartSLOduration=1.05615734 podStartE2EDuration="1.682819779s" totalImagesPullingTime="626.662439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:00:53.681643477 +0000 UTC m=+1551.387964946" watchObservedRunningTime="2026-07-10 15:00:53.682819779 +0000 UTC m=+1551.389141251"
Jul 10 15:00:55.098775 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:55.098728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:57.219250 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.219217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:57.453591 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.453556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:57.550400 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.550286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:57.720465 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.720428    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"]
Jul 10 15:00:57.720751 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.720698    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6" podUID="035bf7e6-f296-4dba-aec7-93ca4bad6ea8" containerName="maas-api" containerID="cri-o://fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46" gracePeriod=30
Jul 10 15:00:57.970753 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.970725    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:57.978507 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:57.978476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:00:58.095203 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.095119    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59r6d\" (UniqueName: \"kubernetes.io/projected/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-kube-api-access-59r6d\") pod \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") "
Jul 10 15:00:58.095203 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.095155    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-openshift-service-ca\") pod \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") "
Jul 10 15:00:58.095203 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.095190    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-maas-api-tls\") pod \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\" (UID: \"035bf7e6-f296-4dba-aec7-93ca4bad6ea8\") "
Jul 10 15:00:58.095498 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.095476    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "035bf7e6-f296-4dba-aec7-93ca4bad6ea8" (UID: "035bf7e6-f296-4dba-aec7-93ca4bad6ea8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:00:58.097362 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.097341    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "035bf7e6-f296-4dba-aec7-93ca4bad6ea8" (UID: "035bf7e6-f296-4dba-aec7-93ca4bad6ea8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:00:58.097451 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.097431    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-kube-api-access-59r6d" (OuterVolumeSpecName: "kube-api-access-59r6d") pod "035bf7e6-f296-4dba-aec7-93ca4bad6ea8" (UID: "035bf7e6-f296-4dba-aec7-93ca4bad6ea8"). InnerVolumeSpecName "kube-api-access-59r6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:00:58.196503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.196470    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:00:58.196503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.196497    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59r6d\" (UniqueName: \"kubernetes.io/projected/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-kube-api-access-59r6d\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:00:58.196711 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.196547    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035bf7e6-f296-4dba-aec7-93ca4bad6ea8-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:00:58.694260 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.694223    2583 generic.go:356] "Generic (PLEG): container finished" podID="035bf7e6-f296-4dba-aec7-93ca4bad6ea8" containerID="fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46" exitCode=0
Jul 10 15:00:58.694696 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.694287    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"
Jul 10 15:00:58.694696 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.694314    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6" event={"ID":"035bf7e6-f296-4dba-aec7-93ca4bad6ea8","Type":"ContainerDied","Data":"fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46"}
Jul 10 15:00:58.694696 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.694374    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6" event={"ID":"035bf7e6-f296-4dba-aec7-93ca4bad6ea8","Type":"ContainerDied","Data":"a39abc466ac332455b19a87b9a9b569f7b105e491d5e7521ab42c4c26a4f47f7"}
Jul 10 15:00:58.694696 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.694395    2583 scope.go:117] "RemoveContainer" containerID="fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46"
Jul 10 15:00:58.704003 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.703986    2583 scope.go:117] "RemoveContainer" containerID="fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46"
Jul 10 15:00:58.704224 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:00:58.704206    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46\": container with ID starting with fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46 not found: ID does not exist" containerID="fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46"
Jul 10 15:00:58.704271 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.704231    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46"} err="failed to get container status \"fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46\": rpc error: code = NotFound desc = could not find container \"fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46\": container with ID starting with fb1d4e8277d4139432d421078399542c28e2951f1d04ce8a907e55dea93d8a46 not found: ID does not exist"
Jul 10 15:00:58.718947 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.718924    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"]
Jul 10 15:00:58.723553 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.723529    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4a6b0771-cfccdfbcd-hbgq6"]
Jul 10 15:00:58.847430 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:58.847399    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="035bf7e6-f296-4dba-aec7-93ca4bad6ea8" path="/var/lib/kubelet/pods/035bf7e6-f296-4dba-aec7-93ca4bad6ea8/volumes"
Jul 10 15:00:59.261568 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:00:59.261493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:00.012828 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:00.012795    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"]
Jul 10 15:01:00.013228 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:00.013000    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz" podUID="40040c28-a114-46c2-af35-bc1632d5bebe" containerName="cleanup" containerID="cri-o://f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3" gracePeriod=30
Jul 10 15:01:04.179933 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:04.179891    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:04.279741 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:04.279711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:04.423024 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:04.422993    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:04.524882 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:04.524803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:05.612127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.612090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:05.686182 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.686155    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"]
Jul 10 15:01:05.686637 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.686622    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="035bf7e6-f296-4dba-aec7-93ca4bad6ea8" containerName="maas-api"
Jul 10 15:01:05.686637 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.686638    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="035bf7e6-f296-4dba-aec7-93ca4bad6ea8" containerName="maas-api"
Jul 10 15:01:05.686797 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.686711    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="035bf7e6-f296-4dba-aec7-93ca4bad6ea8" containerName="maas-api"
Jul 10 15:01:05.690992 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.690972    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.693674 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.693656    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-86b17b7e-serving-cert\""
Jul 10 15:01:05.703121 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.703099    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"]
Jul 10 15:01:05.740125 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.740088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:05.825416 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.825382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:05.869456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.869385    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m88p\" (UniqueName: \"kubernetes.io/projected/9b651e96-c0f4-476d-a1b5-1a7c22699dad-kube-api-access-2m88p\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.869456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.869423    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b651e96-c0f4-476d-a1b5-1a7c22699dad-openshift-service-ca\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.869658 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.869561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.970361 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.970330    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m88p\" (UniqueName: \"kubernetes.io/projected/9b651e96-c0f4-476d-a1b5-1a7c22699dad-kube-api-access-2m88p\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.970503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.970368    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b651e96-c0f4-476d-a1b5-1a7c22699dad-openshift-service-ca\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.970503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.970415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.970628 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:05.970572    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-serving-cert: secret "maas-api-e2e-ait-86b17b7e-serving-cert" not found
Jul 10 15:01:05.970675 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:05.970629    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls podName:9b651e96-c0f4-476d-a1b5-1a7c22699dad nodeName:}" failed. No retries permitted until 2026-07-10 15:01:06.470611964 +0000 UTC m=+1564.176933411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls") pod "maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" (UID: "9b651e96-c0f4-476d-a1b5-1a7c22699dad") : secret "maas-api-e2e-ait-86b17b7e-serving-cert" not found
Jul 10 15:01:05.971061 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.971040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b651e96-c0f4-476d-a1b5-1a7c22699dad-openshift-service-ca\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:05.983818 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:05.983796    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m88p\" (UniqueName: \"kubernetes.io/projected/9b651e96-c0f4-476d-a1b5-1a7c22699dad-kube-api-access-2m88p\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:06.262275 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.262226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:06.476249 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.476215    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:06.478718 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.478682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls\") pod \"maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:06.535155 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.535087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:06.601249 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.601217    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:06.634215 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.634172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:06.764207 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:06.764183    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"]
Jul 10 15:01:06.766790 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:01:06.766762    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b651e96_c0f4_476d_a1b5_1a7c22699dad.slice/crio-ec372e0b73c2ade05468b07d7bd9568b460b51b741296c1e04cfd7f866f64448 WatchSource:0}: Error finding container ec372e0b73c2ade05468b07d7bd9568b460b51b741296c1e04cfd7f866f64448: Status 404 returned error can't find the container with id ec372e0b73c2ade05468b07d7bd9568b460b51b741296c1e04cfd7f866f64448
Jul 10 15:01:07.744995 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:07.744960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" event={"ID":"9b651e96-c0f4-476d-a1b5-1a7c22699dad","Type":"ContainerStarted","Data":"8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883"}
Jul 10 15:01:07.745366 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:07.745038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" event={"ID":"9b651e96-c0f4-476d-a1b5-1a7c22699dad","Type":"ContainerStarted","Data":"ec372e0b73c2ade05468b07d7bd9568b460b51b741296c1e04cfd7f866f64448"}
Jul 10 15:01:07.745366 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:07.745091    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:07.765924 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:07.765879    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" podStartSLOduration=2.134609747 podStartE2EDuration="2.765864578s" totalImagesPullingTime="631.254831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:01:07.764731243 +0000 UTC m=+1565.471052717" watchObservedRunningTime="2026-07-10 15:01:07.765864578 +0000 UTC m=+1565.472186059"
Jul 10 15:01:07.847193 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:07.847160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:08.562069 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:08.562028    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:01:10.899728 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:10.899696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:11.323835 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.323798    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"]
Jul 10 15:01:11.324054 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.324012    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" podUID="9b651e96-c0f4-476d-a1b5-1a7c22699dad" containerName="maas-api" containerID="cri-o://8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883" gracePeriod=30
Jul 10 15:01:11.353165 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.353139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:11.572487 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.572465    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:11.725971 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.725946    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b651e96-c0f4-476d-a1b5-1a7c22699dad-openshift-service-ca\") pod \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") "
Jul 10 15:01:11.726179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.725979    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls\") pod \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") "
Jul 10 15:01:11.726179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.725999    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m88p\" (UniqueName: \"kubernetes.io/projected/9b651e96-c0f4-476d-a1b5-1a7c22699dad-kube-api-access-2m88p\") pod \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\" (UID: \"9b651e96-c0f4-476d-a1b5-1a7c22699dad\") "
Jul 10 15:01:11.726299 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.726274    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b651e96-c0f4-476d-a1b5-1a7c22699dad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b651e96-c0f4-476d-a1b5-1a7c22699dad" (UID: "9b651e96-c0f4-476d-a1b5-1a7c22699dad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:01:11.728035 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.728005    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b651e96-c0f4-476d-a1b5-1a7c22699dad-kube-api-access-2m88p" (OuterVolumeSpecName: "kube-api-access-2m88p") pod "9b651e96-c0f4-476d-a1b5-1a7c22699dad" (UID: "9b651e96-c0f4-476d-a1b5-1a7c22699dad"). InnerVolumeSpecName "kube-api-access-2m88p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:01:11.728133 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.728087    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b651e96-c0f4-476d-a1b5-1a7c22699dad" (UID: "9b651e96-c0f4-476d-a1b5-1a7c22699dad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:01:11.767664 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.767635    2583 generic.go:356] "Generic (PLEG): container finished" podID="9b651e96-c0f4-476d-a1b5-1a7c22699dad" containerID="8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883" exitCode=0
Jul 10 15:01:11.767762 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.767682    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"
Jul 10 15:01:11.767806 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.767684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" event={"ID":"9b651e96-c0f4-476d-a1b5-1a7c22699dad","Type":"ContainerDied","Data":"8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883"}
Jul 10 15:01:11.767842 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.767807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq" event={"ID":"9b651e96-c0f4-476d-a1b5-1a7c22699dad","Type":"ContainerDied","Data":"ec372e0b73c2ade05468b07d7bd9568b460b51b741296c1e04cfd7f866f64448"}
Jul 10 15:01:11.767842 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.767832    2583 scope.go:117] "RemoveContainer" containerID="8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883"
Jul 10 15:01:11.777396 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.777378    2583 scope.go:117] "RemoveContainer" containerID="8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883"
Jul 10 15:01:11.777683 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:11.777666    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883\": container with ID starting with 8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883 not found: ID does not exist" containerID="8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883"
Jul 10 15:01:11.777734 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.777689    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883"} err="failed to get container status \"8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883\": rpc error: code = NotFound desc = could not find container \"8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883\": container with ID starting with 8b1c4508c8ebd26a55695ee01734f0b24ca1369ce1db3d6a855b735341394883 not found: ID does not exist"
Jul 10 15:01:11.793011 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.792910    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"]
Jul 10 15:01:11.794904 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.794889    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86b17b7e-77468d58bb-gqbmq"]
Jul 10 15:01:11.827114 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.827098    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b651e96-c0f4-476d-a1b5-1a7c22699dad-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:01:11.827191 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.827116    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m88p\" (UniqueName: \"kubernetes.io/projected/9b651e96-c0f4-476d-a1b5-1a7c22699dad-kube-api-access-2m88p\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:01:11.827191 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:11.827126    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b651e96-c0f4-476d-a1b5-1a7c22699dad-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:01:12.846358 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:12.846320    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b651e96-c0f4-476d-a1b5-1a7c22699dad" path="/var/lib/kubelet/pods/9b651e96-c0f4-476d-a1b5-1a7c22699dad/volumes"
Jul 10 15:01:16.929099 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:16.929061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:17.030868 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:17.030835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:18.527266 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:18.527229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:19.866714 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:19.866682    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:01:23.928685 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:23.928651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:24.019660 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:24.019616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:24.151417 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:24.151386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:24.247496 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:24.247414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:25.346677 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.346630    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:25.414352 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.414317    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"]
Jul 10 15:01:25.414885 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.414862    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b651e96-c0f4-476d-a1b5-1a7c22699dad" containerName="maas-api"
Jul 10 15:01:25.414885 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.414885    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b651e96-c0f4-476d-a1b5-1a7c22699dad" containerName="maas-api"
Jul 10 15:01:25.415056 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.415046    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b651e96-c0f4-476d-a1b5-1a7c22699dad" containerName="maas-api"
Jul 10 15:01:25.420019 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.419997    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.423644 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.423616    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-11c618da-serving-cert\""
Jul 10 15:01:25.431353 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.431327    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"]
Jul 10 15:01:25.437887 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.437860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v89c9\" (UniqueName: \"kubernetes.io/projected/e66ece61-040c-430f-ade6-7a3ced303299-kube-api-access-v89c9\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.438003 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.437932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66ece61-040c-430f-ade6-7a3ced303299-openshift-service-ca\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.438073 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.438032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.460207 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.460176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:25.539392 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.539356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66ece61-040c-430f-ade6-7a3ced303299-openshift-service-ca\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.539580 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.539422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.539580 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.539492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v89c9\" (UniqueName: \"kubernetes.io/projected/e66ece61-040c-430f-ade6-7a3ced303299-kube-api-access-v89c9\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.539710 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:25.539581    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-serving-cert: secret "maas-api-e2e-derive-11c618da-serving-cert" not found
Jul 10 15:01:25.539710 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:25.539659    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls podName:e66ece61-040c-430f-ade6-7a3ced303299 nodeName:}" failed. No retries permitted until 2026-07-10 15:01:26.039635909 +0000 UTC m=+1583.745957373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls") pod "maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" (UID: "e66ece61-040c-430f-ade6-7a3ced303299") : secret "maas-api-e2e-derive-11c618da-serving-cert" not found
Jul 10 15:01:25.539999 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.539980    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66ece61-040c-430f-ade6-7a3ced303299-openshift-service-ca\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.552113 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.552090    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v89c9\" (UniqueName: \"kubernetes.io/projected/e66ece61-040c-430f-ade6-7a3ced303299-kube-api-access-v89c9\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:25.566467 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:25.566443    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:26.015504 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:26.015467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:26.044422 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:26.044392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:26.044617 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:26.044571    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-serving-cert: secret "maas-api-e2e-derive-11c618da-serving-cert" not found
Jul 10 15:01:26.044688 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:26.044666    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls podName:e66ece61-040c-430f-ade6-7a3ced303299 nodeName:}" failed. No retries permitted until 2026-07-10 15:01:27.044643766 +0000 UTC m=+1584.750965235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls") pod "maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" (UID: "e66ece61-040c-430f-ade6-7a3ced303299") : secret "maas-api-e2e-derive-11c618da-serving-cert" not found
Jul 10 15:01:26.149695 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:26.149647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:26.329611 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:26.329496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:27.054010 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.053971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:27.056338 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.056311    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") pod \"maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:27.232505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.232471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:27.394258 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.394232    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"]
Jul 10 15:01:27.395428 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:01:27.395399    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66ece61_040c_430f_ade6_7a3ced303299.slice/crio-14a2132e01defacd4e576f2662332b0df74d02165fa11aad330b7acd5b022f46 WatchSource:0}: Error finding container 14a2132e01defacd4e576f2662332b0df74d02165fa11aad330b7acd5b022f46: Status 404 returned error can't find the container with id 14a2132e01defacd4e576f2662332b0df74d02165fa11aad330b7acd5b022f46
Jul 10 15:01:27.861264 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.861230    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" event={"ID":"e66ece61-040c-430f-ade6-7a3ced303299","Type":"ContainerStarted","Data":"a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246"}
Jul 10 15:01:27.861434 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.861290    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" event={"ID":"e66ece61-040c-430f-ade6-7a3ced303299","Type":"ContainerStarted","Data":"14a2132e01defacd4e576f2662332b0df74d02165fa11aad330b7acd5b022f46"}
Jul 10 15:01:27.861434 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.861316    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:27.886051 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:27.885999    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" podStartSLOduration=2.529906893 podStartE2EDuration="2.885981956s" totalImagesPullingTime="356.075063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:01:27.882454915 +0000 UTC m=+1585.588776394" watchObservedRunningTime="2026-07-10 15:01:27.885981956 +0000 UTC m=+1585.592303424"
Jul 10 15:01:28.177777 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:28.177737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:29.804435 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:29.804399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:29.983467 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:29.983427    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:30.083647 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.083615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:30.213853 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.213825    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"
Jul 10 15:01:30.332563 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.332465    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"]
Jul 10 15:01:30.332868 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.332715    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" podUID="e66ece61-040c-430f-ade6-7a3ced303299" containerName="maas-api" containerID="cri-o://a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246" gracePeriod=30
Jul 10 15:01:30.388900 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.388865    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:30.582700 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.582637    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:30.687857 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.687827    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66ece61-040c-430f-ade6-7a3ced303299-openshift-service-ca\") pod \"e66ece61-040c-430f-ade6-7a3ced303299\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") "
Jul 10 15:01:30.688040 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.687894    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") pod \"e66ece61-040c-430f-ade6-7a3ced303299\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") "
Jul 10 15:01:30.688040 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.687973    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v89c9\" (UniqueName: \"kubernetes.io/projected/e66ece61-040c-430f-ade6-7a3ced303299-kube-api-access-v89c9\") pod \"e66ece61-040c-430f-ade6-7a3ced303299\" (UID: \"e66ece61-040c-430f-ade6-7a3ced303299\") "
Jul 10 15:01:30.688186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.688164    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e66ece61-040c-430f-ade6-7a3ced303299-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e66ece61-040c-430f-ade6-7a3ced303299" (UID: "e66ece61-040c-430f-ade6-7a3ced303299"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:01:30.689970 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.689946    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e66ece61-040c-430f-ade6-7a3ced303299" (UID: "e66ece61-040c-430f-ade6-7a3ced303299"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:01:30.690042 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.689977    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66ece61-040c-430f-ade6-7a3ced303299-kube-api-access-v89c9" (OuterVolumeSpecName: "kube-api-access-v89c9") pod "e66ece61-040c-430f-ade6-7a3ced303299" (UID: "e66ece61-040c-430f-ade6-7a3ced303299"). InnerVolumeSpecName "kube-api-access-v89c9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:01:30.789388 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.789353    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66ece61-040c-430f-ade6-7a3ced303299-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:01:30.789388 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.789381    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v89c9\" (UniqueName: \"kubernetes.io/projected/e66ece61-040c-430f-ade6-7a3ced303299-kube-api-access-v89c9\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:01:30.789388 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.789391    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66ece61-040c-430f-ade6-7a3ced303299-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:01:30.879816 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.879781    2583 generic.go:356] "Generic (PLEG): container finished" podID="e66ece61-040c-430f-ade6-7a3ced303299" containerID="a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246" exitCode=0
Jul 10 15:01:30.880179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.879828    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"
Jul 10 15:01:30.880179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.879875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" event={"ID":"e66ece61-040c-430f-ade6-7a3ced303299","Type":"ContainerDied","Data":"a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246"}
Jul 10 15:01:30.880179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.879926    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt" event={"ID":"e66ece61-040c-430f-ade6-7a3ced303299","Type":"ContainerDied","Data":"14a2132e01defacd4e576f2662332b0df74d02165fa11aad330b7acd5b022f46"}
Jul 10 15:01:30.880179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.879951    2583 scope.go:117] "RemoveContainer" containerID="a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246"
Jul 10 15:01:30.881224 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.881197    2583 generic.go:356] "Generic (PLEG): container finished" podID="40040c28-a114-46c2-af35-bc1632d5bebe" containerID="f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3" exitCode=137
Jul 10 15:01:30.881320 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.881239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz" event={"ID":"40040c28-a114-46c2-af35-bc1632d5bebe","Type":"ContainerDied","Data":"f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3"}
Jul 10 15:01:30.881320 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.881281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz" event={"ID":"40040c28-a114-46c2-af35-bc1632d5bebe","Type":"ContainerDied","Data":"c02b137e3446d9e5f21edde6c1ae7489da6dea1243a0eeb6d57121e3778adf46"}
Jul 10 15:01:30.881320 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.881241    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"
Jul 10 15:01:30.889033 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.889017    2583 scope.go:117] "RemoveContainer" containerID="a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246"
Jul 10 15:01:30.889292 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:30.889270    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246\": container with ID starting with a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246 not found: ID does not exist" containerID="a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246"
Jul 10 15:01:30.889335 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.889299    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246"} err="failed to get container status \"a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246\": rpc error: code = NotFound desc = could not find container \"a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246\": container with ID starting with a2a27bf7176b3c52b836dffc58f257c14e2a67036f9561dc003b95659bc64246 not found: ID does not exist"
Jul 10 15:01:30.889335 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.889315    2583 scope.go:117] "RemoveContainer" containerID="f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3"
Jul 10 15:01:30.896827 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.896811    2583 scope.go:117] "RemoveContainer" containerID="f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3"
Jul 10 15:01:30.897089 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:01:30.897063    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3\": container with ID starting with f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3 not found: ID does not exist" containerID="f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3"
Jul 10 15:01:30.897128 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.897090    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3"} err="failed to get container status \"f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3\": rpc error: code = NotFound desc = could not find container \"f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3\": container with ID starting with f046cd505abc5790541749b523c19c8b309216f98c87056795375afd9c977bc3 not found: ID does not exist"
Jul 10 15:01:30.905149 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.905130    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"]
Jul 10 15:01:30.909779 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.909759    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728260-xpnbz"]
Jul 10 15:01:30.921170 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.921149    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"]
Jul 10 15:01:30.925645 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:30.925625    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11c618da-6d4786bc5b-jldzt"]
Jul 10 15:01:32.846648 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:32.846617    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40040c28-a114-46c2-af35-bc1632d5bebe" path="/var/lib/kubelet/pods/40040c28-a114-46c2-af35-bc1632d5bebe/volumes"
Jul 10 15:01:32.847018 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:32.846973    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e66ece61-040c-430f-ade6-7a3ced303299" path="/var/lib/kubelet/pods/e66ece61-040c-430f-ade6-7a3ced303299/volumes"
Jul 10 15:01:33.159821 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:33.159731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:33.224824 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:33.224791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:33.387229 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:33.387196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:36.561456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:36.561416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:36.726624 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:36.726583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:37.039234 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:37.039201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:37.139990 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:37.139953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:37.234554 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:37.234486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:38.651386 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:38.651344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:45.203982 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:45.203944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:45.312533 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:45.312485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:45.415809 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:45.415777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:45.522044 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:45.521963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:01:45.983481 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:01:45.983448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:11.935283 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:11.935248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:18.384698 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:18.384663    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:18.486928 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:18.486894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:18.686808 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:18.686772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:34.539575 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:34.539544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:41.397230 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:41.397198    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:41.508855 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:41.508821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:41.709467 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:41.709435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:52.199012 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:52.198979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:52.991064 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:52.991026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:59.569933 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:59.569902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:59.672944 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:59.672907    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:59.790903 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:59.790868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:02:59.896183 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:02:59.896096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:00.814724 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:00.814682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:00.955957 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:00.955926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:02.582969 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:02.582924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:03.182171 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:03.182139    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:03:09.369168 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:09.369129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:09.465427 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:09.465391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:10.069153 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:10.069104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:12.463285 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:12.463254    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:03:17.045447 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:17.045407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:17.156612 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:17.156575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:17.285534 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:17.285474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:17.782214 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:17.782169    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:18.073949 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:18.073861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:20.492663 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.492626    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:20.600872 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.600841    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"]
Jul 10 15:03:20.601317 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.601304    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40040c28-a114-46c2-af35-bc1632d5bebe" containerName="cleanup"
Jul 10 15:03:20.601361 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.601319    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="40040c28-a114-46c2-af35-bc1632d5bebe" containerName="cleanup"
Jul 10 15:03:20.601361 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.601339    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e66ece61-040c-430f-ade6-7a3ced303299" containerName="maas-api"
Jul 10 15:03:20.601361 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.601345    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66ece61-040c-430f-ade6-7a3ced303299" containerName="maas-api"
Jul 10 15:03:20.601450 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.601410    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="40040c28-a114-46c2-af35-bc1632d5bebe" containerName="cleanup"
Jul 10 15:03:20.601450 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.601420    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e66ece61-040c-430f-ade6-7a3ced303299" containerName="maas-api"
Jul 10 15:03:20.604483 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.604465    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.607954 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.607934    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-28b1bf63-serving-cert\""
Jul 10 15:03:20.624465 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.624444    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"]
Jul 10 15:03:20.631239 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.631212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:20.654592 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.654564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.654697 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.654604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8km7n\" (UniqueName: \"kubernetes.io/projected/45eb933e-d221-4fc3-ae9b-2f35f5987e63-kube-api-access-8km7n\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.654787 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.654773    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45eb933e-d221-4fc3-ae9b-2f35f5987e63-openshift-service-ca\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.748582 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.748487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:20.755630 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.755607    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45eb933e-d221-4fc3-ae9b-2f35f5987e63-openshift-service-ca\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.755727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.755637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.755727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.755657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8km7n\" (UniqueName: \"kubernetes.io/projected/45eb933e-d221-4fc3-ae9b-2f35f5987e63-kube-api-access-8km7n\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.755833 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:03:20.755767    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-serving-cert: secret "maas-api-e2e-mt-28b1bf63-serving-cert" not found
Jul 10 15:03:20.755833 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:03:20.755828    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls podName:45eb933e-d221-4fc3-ae9b-2f35f5987e63 nodeName:}" failed. No retries permitted until 2026-07-10 15:03:21.255811062 +0000 UTC m=+1698.962132528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls") pod "maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" (UID: "45eb933e-d221-4fc3-ae9b-2f35f5987e63") : secret "maas-api-e2e-mt-28b1bf63-serving-cert" not found
Jul 10 15:03:20.756260 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.756238    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45eb933e-d221-4fc3-ae9b-2f35f5987e63-openshift-service-ca\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.768289 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.768268    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8km7n\" (UniqueName: \"kubernetes.io/projected/45eb933e-d221-4fc3-ae9b-2f35f5987e63-kube-api-access-8km7n\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:20.878853 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:20.878826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:21.260916 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:21.260878    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:21.263292 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:21.263259    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls\") pod \"maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:21.315333 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:21.315291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:21.412969 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:21.412937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:21.514875 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:21.514795    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:21.658602 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:21.658565    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"]
Jul 10 15:03:21.660397 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:03:21.660369    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45eb933e_d221_4fc3_ae9b_2f35f5987e63.slice/crio-6ae256ea8528f5003ddaf47827d6ebbf93547dcf8e5d76f7b4f1a56abe1e5031 WatchSource:0}: Error finding container 6ae256ea8528f5003ddaf47827d6ebbf93547dcf8e5d76f7b4f1a56abe1e5031: Status 404 returned error can't find the container with id 6ae256ea8528f5003ddaf47827d6ebbf93547dcf8e5d76f7b4f1a56abe1e5031
Jul 10 15:03:22.457816 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:22.457780    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" event={"ID":"45eb933e-d221-4fc3-ae9b-2f35f5987e63","Type":"ContainerStarted","Data":"55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0"}
Jul 10 15:03:22.458001 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:22.457833    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" event={"ID":"45eb933e-d221-4fc3-ae9b-2f35f5987e63","Type":"ContainerStarted","Data":"6ae256ea8528f5003ddaf47827d6ebbf93547dcf8e5d76f7b4f1a56abe1e5031"}
Jul 10 15:03:22.458001 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:22.457881    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:22.491703 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:22.491659    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" podStartSLOduration=2.070789522 podStartE2EDuration="2.491644968s" totalImagesPullingTime="420.855446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:03:22.490171306 +0000 UTC m=+1700.196492774" watchObservedRunningTime="2026-07-10 15:03:22.491644968 +0000 UTC m=+1700.197966435"
Jul 10 15:03:24.881502 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:24.881414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:25.544349 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:25.544301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:25.703724 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:25.703684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:25.964369 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:25.964322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:26.593204 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:26.593167    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:03:28.468456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:28.468424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:33.130774 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:33.130741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:33.277678 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:33.277642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:33.558509 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:33.558473    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:33.905957 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:33.905868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:33.990992 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:33.990955    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"]
Jul 10 15:03:33.991202 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:33.991180    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" podUID="45eb933e-d221-4fc3-ae9b-2f35f5987e63" containerName="maas-api" containerID="cri-o://55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0" gracePeriod=30
Jul 10 15:03:34.024792 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.024764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:34.240711 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.240688    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:34.375757 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.375730    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8km7n\" (UniqueName: \"kubernetes.io/projected/45eb933e-d221-4fc3-ae9b-2f35f5987e63-kube-api-access-8km7n\") pod \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") "
Jul 10 15:03:34.375925 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.375792    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45eb933e-d221-4fc3-ae9b-2f35f5987e63-openshift-service-ca\") pod \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") "
Jul 10 15:03:34.375925 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.375866    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls\") pod \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\" (UID: \"45eb933e-d221-4fc3-ae9b-2f35f5987e63\") "
Jul 10 15:03:34.376186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.376158    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45eb933e-d221-4fc3-ae9b-2f35f5987e63-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45eb933e-d221-4fc3-ae9b-2f35f5987e63" (UID: "45eb933e-d221-4fc3-ae9b-2f35f5987e63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:03:34.377873 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.377849    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45eb933e-d221-4fc3-ae9b-2f35f5987e63-kube-api-access-8km7n" (OuterVolumeSpecName: "kube-api-access-8km7n") pod "45eb933e-d221-4fc3-ae9b-2f35f5987e63" (UID: "45eb933e-d221-4fc3-ae9b-2f35f5987e63"). InnerVolumeSpecName "kube-api-access-8km7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:03:34.377955 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.377875    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45eb933e-d221-4fc3-ae9b-2f35f5987e63" (UID: "45eb933e-d221-4fc3-ae9b-2f35f5987e63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:03:34.476935 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.476905    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45eb933e-d221-4fc3-ae9b-2f35f5987e63-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:03:34.476935 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.476931    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8km7n\" (UniqueName: \"kubernetes.io/projected/45eb933e-d221-4fc3-ae9b-2f35f5987e63-kube-api-access-8km7n\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:03:34.476935 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.476940    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45eb933e-d221-4fc3-ae9b-2f35f5987e63-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:03:34.518671 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.518636    2583 generic.go:356] "Generic (PLEG): container finished" podID="45eb933e-d221-4fc3-ae9b-2f35f5987e63" containerID="55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0" exitCode=0
Jul 10 15:03:34.518832 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.518677    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"
Jul 10 15:03:34.518832 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.518719    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" event={"ID":"45eb933e-d221-4fc3-ae9b-2f35f5987e63","Type":"ContainerDied","Data":"55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0"}
Jul 10 15:03:34.518832 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.518767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2" event={"ID":"45eb933e-d221-4fc3-ae9b-2f35f5987e63","Type":"ContainerDied","Data":"6ae256ea8528f5003ddaf47827d6ebbf93547dcf8e5d76f7b4f1a56abe1e5031"}
Jul 10 15:03:34.518832 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.518791    2583 scope.go:117] "RemoveContainer" containerID="55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0"
Jul 10 15:03:34.528368 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.528351    2583 scope.go:117] "RemoveContainer" containerID="55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0"
Jul 10 15:03:34.528643 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:03:34.528625    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0\": container with ID starting with 55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0 not found: ID does not exist" containerID="55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0"
Jul 10 15:03:34.528708 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.528652    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0"} err="failed to get container status \"55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0\": rpc error: code = NotFound desc = could not find container \"55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0\": container with ID starting with 55a65e68c566b678726cf2235aa3add32aa5c71eed754af6fb38a9254934c7d0 not found: ID does not exist"
Jul 10 15:03:34.543862 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.543836    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"]
Jul 10 15:03:34.556894 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.556871    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b1bf63-7f58658757-7bdm2"]
Jul 10 15:03:34.846600 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:34.846483    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45eb933e-d221-4fc3-ae9b-2f35f5987e63" path="/var/lib/kubelet/pods/45eb933e-d221-4fc3-ae9b-2f35f5987e63/volumes"
Jul 10 15:03:36.585627 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:36.585593    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:03:40.962550 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:40.962501    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:41.069564 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:41.069498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:41.957947 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:41.957911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:42.061725 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:42.061696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:48.967721 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:48.967679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:49.127971 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:49.127933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:49.231708 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:49.231618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:49.904370 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:49.904335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:56.816858 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:56.816816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:03:56.963045 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:03:56.963006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:04.879479 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:04.879434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:05.354825 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:05.354789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:05.544668 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:05.544631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:07.140743 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:07.140690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:07.313413 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:07.313382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:08.757198 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.757157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:08.881422 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.881385    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"]
Jul 10 15:04:08.881872 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.881857    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45eb933e-d221-4fc3-ae9b-2f35f5987e63" containerName="maas-api"
Jul 10 15:04:08.881929 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.881874    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="45eb933e-d221-4fc3-ae9b-2f35f5987e63" containerName="maas-api"
Jul 10 15:04:08.881965 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.881950    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="45eb933e-d221-4fc3-ae9b-2f35f5987e63" containerName="maas-api"
Jul 10 15:04:08.884938 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.884908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:08.885344 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.885326    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:08.889871 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.889850    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ecb19c5e-serving-cert\""
Jul 10 15:04:08.902937 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.902909    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"]
Jul 10 15:04:08.976632 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.976596    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-openshift-service-ca\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:08.976796 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.976729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t252w\" (UniqueName: \"kubernetes.io/projected/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-kube-api-access-t252w\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:08.976796 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.976778    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:08.990217 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:08.990176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:09.077563 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.077446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t252w\" (UniqueName: \"kubernetes.io/projected/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-kube-api-access-t252w\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.077563 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.077494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.077786 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.077591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-openshift-service-ca\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.077786 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:04:09.077673    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-serving-cert: secret "maas-api-e2e-mt-ecb19c5e-serving-cert" not found
Jul 10 15:04:09.077786 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:04:09.077747    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls podName:cfd7b0b2-f740-4ea9-a70e-992bf24111a1 nodeName:}" failed. No retries permitted until 2026-07-10 15:04:09.577729958 +0000 UTC m=+1747.284051410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls") pod "maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" (UID: "cfd7b0b2-f740-4ea9-a70e-992bf24111a1") : secret "maas-api-e2e-mt-ecb19c5e-serving-cert" not found
Jul 10 15:04:09.078165 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.078148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-openshift-service-ca\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.087589 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.087561    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t252w\" (UniqueName: \"kubernetes.io/projected/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-kube-api-access-t252w\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.503971 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.503936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:09.582585 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.582547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.584926 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.584903    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls\") pod \"maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.602693 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.602662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:09.795477 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.795388    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:09.946026 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:09.945975    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"]
Jul 10 15:04:09.948969 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:04:09.948940    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd7b0b2_f740_4ea9_a70e_992bf24111a1.slice/crio-74013ed5ff8747e47fe64dd325b644ae02b389d5e5b272741269eb20d88e0516 WatchSource:0}: Error finding container 74013ed5ff8747e47fe64dd325b644ae02b389d5e5b272741269eb20d88e0516: Status 404 returned error can't find the container with id 74013ed5ff8747e47fe64dd325b644ae02b389d5e5b272741269eb20d88e0516
Jul 10 15:04:10.707786 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:10.707746    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" event={"ID":"cfd7b0b2-f740-4ea9-a70e-992bf24111a1","Type":"ContainerStarted","Data":"74013ed5ff8747e47fe64dd325b644ae02b389d5e5b272741269eb20d88e0516"}
Jul 10 15:04:11.713835 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:11.713792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" event={"ID":"cfd7b0b2-f740-4ea9-a70e-992bf24111a1","Type":"ContainerStarted","Data":"1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976"}
Jul 10 15:04:11.714271 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:11.713856    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:11.738260 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:11.738215    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" podStartSLOduration=2.858365103 podStartE2EDuration="3.73819365s" totalImagesPullingTime="879.828547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:04:11.73735148 +0000 UTC m=+1749.443672945" watchObservedRunningTime="2026-07-10 15:04:11.73819365 +0000 UTC m=+1749.444515097"
Jul 10 15:04:13.183907 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:13.183866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:15.675770 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:15.675732    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:04:17.724896 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:17.724862    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:30.011353 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.011323    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"]
Jul 10 15:04:30.015077 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.015061    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.018119 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.018096    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ecb19c5e\"/\"e2e-lifecycle-model-ecb19c5e-kserve-self-signed-certs\""
Jul 10 15:04:30.019393 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.019375    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ecb19c5e\"/\"default-dockercfg-4dnw8\""
Jul 10 15:04:30.019529 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.019392    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ecb19c5e\"/\"kube-root-ca.crt\""
Jul 10 15:04:30.019529 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.019453    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ecb19c5e\"/\"openshift-service-ca.crt\""
Jul 10 15:04:30.029983 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.029964    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"]
Jul 10 15:04:30.080179 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.080132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:30.168845 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.168818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-home\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.168978 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.168855    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-tmp-dir\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.168978 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.168912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zkrx\" (UniqueName: \"kubernetes.io/projected/3da5d2c0-1f39-4526-bcae-94f376249614-kube-api-access-6zkrx\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.168978 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.168934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-model-cache\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.168978 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.168965    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da5d2c0-1f39-4526-bcae-94f376249614-tls-certs\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.169166 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.168988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-dshm\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270457 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zkrx\" (UniqueName: \"kubernetes.io/projected/3da5d2c0-1f39-4526-bcae-94f376249614-kube-api-access-6zkrx\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270457 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-model-cache\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270652 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da5d2c0-1f39-4526-bcae-94f376249614-tls-certs\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270700 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-dshm\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270736 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-home\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270776 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270743    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-tmp-dir\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.270842 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.270825    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-model-cache\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.271068 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.271052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-tmp-dir\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.271134 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.271068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-home\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.272851 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.272830    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-dshm\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.273150 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.273131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da5d2c0-1f39-4526-bcae-94f376249614-tls-certs\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.279940 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.279917    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zkrx\" (UniqueName: \"kubernetes.io/projected/3da5d2c0-1f39-4526-bcae-94f376249614-kube-api-access-6zkrx\") pod \"e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") " pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.324965 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.324940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:30.465174 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.465049    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"]
Jul 10 15:04:30.467994 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:04:30.467971    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3da5d2c0_1f39_4526_bcae_94f376249614.slice/crio-bb550218eda987895098037fb573ce51fcd42b37d35d11510b83bc6f9a562a43 WatchSource:0}: Error finding container bb550218eda987895098037fb573ce51fcd42b37d35d11510b83bc6f9a562a43: Status 404 returned error can't find the container with id bb550218eda987895098037fb573ce51fcd42b37d35d11510b83bc6f9a562a43
Jul 10 15:04:30.810534 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.810440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv" event={"ID":"3da5d2c0-1f39-4526-bcae-94f376249614","Type":"ContainerStarted","Data":"8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c"}
Jul 10 15:04:30.810534 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.810478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv" event={"ID":"3da5d2c0-1f39-4526-bcae-94f376249614","Type":"ContainerStarted","Data":"bb550218eda987895098037fb573ce51fcd42b37d35d11510b83bc6f9a562a43"}
Jul 10 15:04:30.840537 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:30.833863    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv" podStartSLOduration=1.833843189 podStartE2EDuration="1.833843189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:04:30.831659865 +0000 UTC m=+1768.537981331" watchObservedRunningTime="2026-07-10 15:04:30.833843189 +0000 UTC m=+1768.540164658"
Jul 10 15:04:33.366693 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:33.366651    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:04:40.325265 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:40.325216    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:40.325716 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:40.325280    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:40.339049 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:40.339020    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:40.875442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:40.875413    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:04:48.177471 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:48.177436    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:04:50.720675 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:50.720625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:51.295363 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.295330    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"]
Jul 10 15:04:51.295580 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.295555    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" podUID="cfd7b0b2-f740-4ea9-a70e-992bf24111a1" containerName="maas-api" containerID="cri-o://1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976" gracePeriod=30
Jul 10 15:04:51.332730 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.332673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:04:51.549935 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.549867    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:51.664887 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.664849    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-openshift-service-ca\") pod \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") "
Jul 10 15:04:51.664887 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.664889    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t252w\" (UniqueName: \"kubernetes.io/projected/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-kube-api-access-t252w\") pod \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") "
Jul 10 15:04:51.665150 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.664909    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls\") pod \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\" (UID: \"cfd7b0b2-f740-4ea9-a70e-992bf24111a1\") "
Jul 10 15:04:51.665386 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.665356    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfd7b0b2-f740-4ea9-a70e-992bf24111a1" (UID: "cfd7b0b2-f740-4ea9-a70e-992bf24111a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:04:51.667114 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.667083    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfd7b0b2-f740-4ea9-a70e-992bf24111a1" (UID: "cfd7b0b2-f740-4ea9-a70e-992bf24111a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:04:51.667186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.667094    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-kube-api-access-t252w" (OuterVolumeSpecName: "kube-api-access-t252w") pod "cfd7b0b2-f740-4ea9-a70e-992bf24111a1" (UID: "cfd7b0b2-f740-4ea9-a70e-992bf24111a1"). InnerVolumeSpecName "kube-api-access-t252w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:04:51.766226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.766185    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:04:51.766226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.766221    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t252w\" (UniqueName: \"kubernetes.io/projected/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-kube-api-access-t252w\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:04:51.766226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.766234    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd7b0b2-f740-4ea9-a70e-992bf24111a1-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:04:51.918451 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.918416    2583 generic.go:356] "Generic (PLEG): container finished" podID="cfd7b0b2-f740-4ea9-a70e-992bf24111a1" containerID="1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976" exitCode=0
Jul 10 15:04:51.918626 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.918466    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"
Jul 10 15:04:51.918626 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.918507    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" event={"ID":"cfd7b0b2-f740-4ea9-a70e-992bf24111a1","Type":"ContainerDied","Data":"1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976"}
Jul 10 15:04:51.918626 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.918602    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6" event={"ID":"cfd7b0b2-f740-4ea9-a70e-992bf24111a1","Type":"ContainerDied","Data":"74013ed5ff8747e47fe64dd325b644ae02b389d5e5b272741269eb20d88e0516"}
Jul 10 15:04:51.918730 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.918629    2583 scope.go:117] "RemoveContainer" containerID="1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976"
Jul 10 15:04:51.928081 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.928061    2583 scope.go:117] "RemoveContainer" containerID="1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976"
Jul 10 15:04:51.928339 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:04:51.928319    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976\": container with ID starting with 1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976 not found: ID does not exist" containerID="1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976"
Jul 10 15:04:51.928406 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.928347    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976"} err="failed to get container status \"1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976\": rpc error: code = NotFound desc = could not find container \"1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976\": container with ID starting with 1b86b41615c8f5ab3c0c4858d5e24aa52595952bff12b2821033ae7f7c643976 not found: ID does not exist"
Jul 10 15:04:51.944348 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.944326    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"]
Jul 10 15:04:51.951503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:51.951483    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ecb19c5e-d9f9d45b4-fztk6"]
Jul 10 15:04:52.847013 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:52.846980    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd7b0b2-f740-4ea9-a70e-992bf24111a1" path="/var/lib/kubelet/pods/cfd7b0b2-f740-4ea9-a70e-992bf24111a1/volumes"
Jul 10 15:04:56.786643 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:04:56.786561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:01.767426 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:01.767390    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"]
Jul 10 15:05:01.861226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:01.861187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:02.281356 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:02.281322    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:05:03.116759 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:03.116726    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:05:03.122996 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:03.122972    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:05:16.787847 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:16.787805    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv" podUID="3da5d2c0-1f39-4526-bcae-94f376249614" containerName="main" containerID="cri-o://8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c" gracePeriod=45
Jul 10 15:05:17.046906 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.046847    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:05:17.052334 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.052299    2583 generic.go:356] "Generic (PLEG): container finished" podID="3da5d2c0-1f39-4526-bcae-94f376249614" containerID="8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c" exitCode=0
Jul 10 15:05:17.052492 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.052342    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"
Jul 10 15:05:17.052492 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.052350    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv" event={"ID":"3da5d2c0-1f39-4526-bcae-94f376249614","Type":"ContainerDied","Data":"8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c"}
Jul 10 15:05:17.052492 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.052379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv" event={"ID":"3da5d2c0-1f39-4526-bcae-94f376249614","Type":"ContainerDied","Data":"bb550218eda987895098037fb573ce51fcd42b37d35d11510b83bc6f9a562a43"}
Jul 10 15:05:17.052492 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.052395    2583 scope.go:117] "RemoveContainer" containerID="8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c"
Jul 10 15:05:17.060929 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.060902    2583 scope.go:117] "RemoveContainer" containerID="8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c"
Jul 10 15:05:17.061155 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:05:17.061135    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c\": container with ID starting with 8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c not found: ID does not exist" containerID="8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c"
Jul 10 15:05:17.061204 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.061163    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c"} err="failed to get container status \"8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c\": rpc error: code = NotFound desc = could not find container \"8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c\": container with ID starting with 8b910981f3d47da57d7c100189bcb3af5566d2e92f8287362be86333bece5e7c not found: ID does not exist"
Jul 10 15:05:17.194794 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.194764    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-tmp-dir\") pod \"3da5d2c0-1f39-4526-bcae-94f376249614\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") "
Jul 10 15:05:17.194979 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.194837    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zkrx\" (UniqueName: \"kubernetes.io/projected/3da5d2c0-1f39-4526-bcae-94f376249614-kube-api-access-6zkrx\") pod \"3da5d2c0-1f39-4526-bcae-94f376249614\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") "
Jul 10 15:05:17.194979 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.194856    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-model-cache\") pod \"3da5d2c0-1f39-4526-bcae-94f376249614\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") "
Jul 10 15:05:17.194979 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.194874    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da5d2c0-1f39-4526-bcae-94f376249614-tls-certs\") pod \"3da5d2c0-1f39-4526-bcae-94f376249614\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") "
Jul 10 15:05:17.194979 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.194911    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-home\") pod \"3da5d2c0-1f39-4526-bcae-94f376249614\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") "
Jul 10 15:05:17.194979 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.194932    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-dshm\") pod \"3da5d2c0-1f39-4526-bcae-94f376249614\" (UID: \"3da5d2c0-1f39-4526-bcae-94f376249614\") "
Jul 10 15:05:17.195307 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.195033    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3da5d2c0-1f39-4526-bcae-94f376249614" (UID: "3da5d2c0-1f39-4526-bcae-94f376249614"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:05:17.195307 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.195151    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-home" (OuterVolumeSpecName: "home") pod "3da5d2c0-1f39-4526-bcae-94f376249614" (UID: "3da5d2c0-1f39-4526-bcae-94f376249614"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:05:17.195307 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.195158    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-model-cache" (OuterVolumeSpecName: "model-cache") pod "3da5d2c0-1f39-4526-bcae-94f376249614" (UID: "3da5d2c0-1f39-4526-bcae-94f376249614"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:05:17.195307 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.195241    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-model-cache\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:05:17.195307 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.195260    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-home\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:05:17.195307 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.195272    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-tmp-dir\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:05:17.197131 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.197105    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-dshm" (OuterVolumeSpecName: "dshm") pod "3da5d2c0-1f39-4526-bcae-94f376249614" (UID: "3da5d2c0-1f39-4526-bcae-94f376249614"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:05:17.197207 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.197145    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da5d2c0-1f39-4526-bcae-94f376249614-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3da5d2c0-1f39-4526-bcae-94f376249614" (UID: "3da5d2c0-1f39-4526-bcae-94f376249614"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:05:17.197258 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.197239    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3da5d2c0-1f39-4526-bcae-94f376249614-kube-api-access-6zkrx" (OuterVolumeSpecName: "kube-api-access-6zkrx") pod "3da5d2c0-1f39-4526-bcae-94f376249614" (UID: "3da5d2c0-1f39-4526-bcae-94f376249614"). InnerVolumeSpecName "kube-api-access-6zkrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:05:17.295764 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.295730    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zkrx\" (UniqueName: \"kubernetes.io/projected/3da5d2c0-1f39-4526-bcae-94f376249614-kube-api-access-6zkrx\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:05:17.295764 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.295758    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da5d2c0-1f39-4526-bcae-94f376249614-tls-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:05:17.295764 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.295767    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da5d2c0-1f39-4526-bcae-94f376249614-dshm\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:05:17.381545 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.381484    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"]
Jul 10 15:05:17.384032 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:17.384007    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ecb19c5e/e2e-lifecycle-model-ecb19c5e-kserve-bfbc4b74c-7rmcv"]
Jul 10 15:05:18.847192 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:18.847156    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3da5d2c0-1f39-4526-bcae-94f376249614" path="/var/lib/kubelet/pods/3da5d2c0-1f39-4526-bcae-94f376249614/volumes"
Jul 10 15:05:37.982045 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:37.982007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:39.580318 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:39.580285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:45.298165 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:45.298129    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:05:47.009657 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:47.009622    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:54.061478 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:54.061423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:54.196106 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:54.196072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:54.411349 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:54.411314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:54.704555 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:54.704448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:55.849685 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:55.849642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:55.997625 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:55.997585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:56.825850 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:56.825802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:56.930394 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:56.930363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:58.453298 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:58.453250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:05:58.586215 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:05:58.586178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:06:05.748123 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:05.748081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:05.866424 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:05.866382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:06.499618 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:06.499579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:10.888372 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:10.888335    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:06:13.416879 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:13.416843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:13.532241 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:13.532205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:13.635787 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:13.635745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:24.141202 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:24.141111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:24.472165 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:24.472130    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:34.412048 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:34.411998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:34.487136 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:34.487098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:45.415858 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:45.415817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:52.017624 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:52.017583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:52.117856 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:52.117818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:52.353329 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:52.353226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:52.698752 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:52.698688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:52.847300 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:52.847269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:55.556476 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.556430    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:55.671279 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.671210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:55.772377 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772344    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"]
Jul 10 15:06:55.772808 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772795    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd7b0b2-f740-4ea9-a70e-992bf24111a1" containerName="maas-api"
Jul 10 15:06:55.772863 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772810    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd7b0b2-f740-4ea9-a70e-992bf24111a1" containerName="maas-api"
Jul 10 15:06:55.772863 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772829    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3da5d2c0-1f39-4526-bcae-94f376249614" containerName="main"
Jul 10 15:06:55.772863 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772835    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3da5d2c0-1f39-4526-bcae-94f376249614" containerName="main"
Jul 10 15:06:55.772956 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772892    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd7b0b2-f740-4ea9-a70e-992bf24111a1" containerName="maas-api"
Jul 10 15:06:55.772956 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.772902    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3da5d2c0-1f39-4526-bcae-94f376249614" containerName="main"
Jul 10 15:06:55.776017 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.776000    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.779570 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.779546    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e6d308-serving-cert\""
Jul 10 15:06:55.789116 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.789093    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"]
Jul 10 15:06:55.830669 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.830588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:55.859603 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.859569    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp9t7\" (UniqueName: \"kubernetes.io/projected/52ce3e86-db61-4f94-8d8d-5143d594264d-kube-api-access-hp9t7\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.859759 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.859687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.859759 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.859714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52ce3e86-db61-4f94-8d8d-5143d594264d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.960995 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.960956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp9t7\" (UniqueName: \"kubernetes.io/projected/52ce3e86-db61-4f94-8d8d-5143d594264d-kube-api-access-hp9t7\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.961174 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.961031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.961174 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.961054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52ce3e86-db61-4f94-8d8d-5143d594264d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.961174 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:06:55.961162    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-serving-cert: secret "maas-api-e2e-inf-a-e6d308-serving-cert" not found
Jul 10 15:06:55.961304 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:06:55.961237    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls podName:52ce3e86-db61-4f94-8d8d-5143d594264d nodeName:}" failed. No retries permitted until 2026-07-10 15:06:56.461218753 +0000 UTC m=+1914.167540210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls") pod "maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" (UID: "52ce3e86-db61-4f94-8d8d-5143d594264d") : secret "maas-api-e2e-inf-a-e6d308-serving-cert" not found
Jul 10 15:06:55.961668 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.961652    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52ce3e86-db61-4f94-8d8d-5143d594264d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:55.974260 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:55.974236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp9t7\" (UniqueName: \"kubernetes.io/projected/52ce3e86-db61-4f94-8d8d-5143d594264d-kube-api-access-hp9t7\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:56.186723 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.186687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:56.456730 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.456624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:56.465708 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.465684    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:56.468247 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.468218    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e6d308-7779c79977-g55ch\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:56.559509 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.559477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:06:56.686672 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.686639    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:56.830654 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.830627    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"]
Jul 10 15:06:56.832927 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:06:56.832888    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ce3e86_db61_4f94_8d8d_5143d594264d.slice/crio-a7b10be75c7f49253e9f6228899a21d0e98803bacb0f167587f314b0b2fb12b0 WatchSource:0}: Error finding container a7b10be75c7f49253e9f6228899a21d0e98803bacb0f167587f314b0b2fb12b0: Status 404 returned error can't find the container with id a7b10be75c7f49253e9f6228899a21d0e98803bacb0f167587f314b0b2fb12b0
Jul 10 15:06:56.834383 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:56.834364    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 15:06:57.581815 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:57.581734    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" event={"ID":"52ce3e86-db61-4f94-8d8d-5143d594264d","Type":"ContainerStarted","Data":"9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0"}
Jul 10 15:06:57.581815 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:57.581787    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" event={"ID":"52ce3e86-db61-4f94-8d8d-5143d594264d","Type":"ContainerStarted","Data":"a7b10be75c7f49253e9f6228899a21d0e98803bacb0f167587f314b0b2fb12b0"}
Jul 10 15:06:57.582208 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:57.581869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:06:57.602198 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:06:57.602157    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" podStartSLOduration=2.13129556 podStartE2EDuration="2.602143418s" totalImagesPullingTime="470.847858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:06:57.601286128 +0000 UTC m=+1915.307607597" watchObservedRunningTime="2026-07-10 15:06:57.602143418 +0000 UTC m=+1915.308464911"
Jul 10 15:07:00.045564 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:00.045500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:00.340573 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:00.340458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:00.741795 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:00.741754    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:00.934450 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:00.934415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:03.379295 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.379258    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:07:03.592944 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.592910    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:07:03.653341 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.653253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:03.716635 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.716602    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"]
Jul 10 15:07:03.721087 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.721067    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.724291 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.724270    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ff844d-serving-cert\""
Jul 10 15:07:03.734012 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.733992    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"]
Jul 10 15:07:03.750305 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.750270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:03.832341 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.832311    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eff4d6-3404-4310-89f4-9ac319d66161-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.832547 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.832381    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjqmv\" (UniqueName: \"kubernetes.io/projected/78eff4d6-3404-4310-89f4-9ac319d66161-kube-api-access-cjqmv\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.832547 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.832426    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.848492 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.848449    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:03.933086 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.933053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjqmv\" (UniqueName: \"kubernetes.io/projected/78eff4d6-3404-4310-89f4-9ac319d66161-kube-api-access-cjqmv\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.933228 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.933101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.933228 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.933137    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eff4d6-3404-4310-89f4-9ac319d66161-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.933299 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:07:03.933254    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-serving-cert: secret "maas-api-e2e-inf-b-ff844d-serving-cert" not found
Jul 10 15:07:03.933336 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:07:03.933322    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls podName:78eff4d6-3404-4310-89f4-9ac319d66161 nodeName:}" failed. No retries permitted until 2026-07-10 15:07:04.433302622 +0000 UTC m=+1922.139624073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls") pod "maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" (UID: "78eff4d6-3404-4310-89f4-9ac319d66161") : secret "maas-api-e2e-inf-b-ff844d-serving-cert" not found
Jul 10 15:07:03.933784 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.933766    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eff4d6-3404-4310-89f4-9ac319d66161-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:03.945124 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:03.945099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjqmv\" (UniqueName: \"kubernetes.io/projected/78eff4d6-3404-4310-89f4-9ac319d66161-kube-api-access-cjqmv\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:04.087556 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.087500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:04.376460 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.376362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:04.438100 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.438067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:04.440671 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.440645    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:04.476456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.476415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:04.588018 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.587982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:04.631927 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.631833    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:04.704364 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.704327    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:04.787563 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:04.787442    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"]
Jul 10 15:07:04.790306 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:07:04.790275    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78eff4d6_3404_4310_89f4_9ac319d66161.slice/crio-f5dbf4609218bf6b7f5b9b0afa349ff3d475252f39c556e17e0d1f70fe8be72e WatchSource:0}: Error finding container f5dbf4609218bf6b7f5b9b0afa349ff3d475252f39c556e17e0d1f70fe8be72e: Status 404 returned error can't find the container with id f5dbf4609218bf6b7f5b9b0afa349ff3d475252f39c556e17e0d1f70fe8be72e
Jul 10 15:07:05.628054 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:05.628021    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" event={"ID":"78eff4d6-3404-4310-89f4-9ac319d66161","Type":"ContainerStarted","Data":"92f729015bd5005f4361d95140f4b5b3a32d602d8c9826219ee5a4c09c21cb0c"}
Jul 10 15:07:05.628412 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:05.628073    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" event={"ID":"78eff4d6-3404-4310-89f4-9ac319d66161","Type":"ContainerStarted","Data":"f5dbf4609218bf6b7f5b9b0afa349ff3d475252f39c556e17e0d1f70fe8be72e"}
Jul 10 15:07:05.628412 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:05.628112    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:05.650362 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:05.650315    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" podStartSLOduration=1.886527995 podStartE2EDuration="2.650299806s" totalImagesPullingTime="763.771811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:07:05.64813669 +0000 UTC m=+1923.354458158" watchObservedRunningTime="2026-07-10 15:07:05.650299806 +0000 UTC m=+1923.356621308"
Jul 10 15:07:08.005281 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:08.005245    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:11.638254 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:11.638207    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:07:14.192306 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.192273    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"]
Jul 10 15:07:14.196117 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.196100    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.208142 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.208110    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e6d308\"/\"default-dockercfg-t8bbs\""
Jul 10 15:07:14.208442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.208421    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e6d308\"/\"openshift-service-ca.crt\""
Jul 10 15:07:14.209359 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.209334    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e6d308\"/\"test-model-e6d308-kserve-self-signed-certs\""
Jul 10 15:07:14.209676 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.209657    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e6d308\"/\"kube-root-ca.crt\""
Jul 10 15:07:14.212901 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.212876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-model-cache\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.213012 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.212910    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-dshm\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.213012 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.212947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zjk9\" (UniqueName: \"kubernetes.io/projected/b5121aaa-a00f-40d5-bd6c-809bac73f938-kube-api-access-7zjk9\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.213145 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.213110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-home\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.213221 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.213199    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5121aaa-a00f-40d5-bd6c-809bac73f938-tls-certs\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.213370 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.213342    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-tmp-dir\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.217105 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.217084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"]
Jul 10 15:07:14.232544 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.232476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:14.313825 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.313791    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-tmp-dir\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314020 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.313843    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-model-cache\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314020 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.313869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-dshm\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314020 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.313911    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zjk9\" (UniqueName: \"kubernetes.io/projected/b5121aaa-a00f-40d5-bd6c-809bac73f938-kube-api-access-7zjk9\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314020 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.313956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-home\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314020 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.313989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5121aaa-a00f-40d5-bd6c-809bac73f938-tls-certs\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314317 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.314262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-tmp-dir\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314317 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.314267    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-model-cache\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.314433 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.314324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-home\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.316269 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.316245    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-dshm\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.316468 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.316449    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5121aaa-a00f-40d5-bd6c-809bac73f938-tls-certs\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.324291 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.324271    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zjk9\" (UniqueName: \"kubernetes.io/projected/b5121aaa-a00f-40d5-bd6c-809bac73f938-kube-api-access-7zjk9\") pod \"test-model-e6d308-kserve-68dc5f49f7-xhf8t\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") " pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.510379 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.510280    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:14.656175 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.656142    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"]
Jul 10 15:07:14.657131 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:07:14.657102    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5121aaa_a00f_40d5_bd6c_809bac73f938.slice/crio-e9e9d85e04cf21c42d3e7210cad41ede3ce21f226d4034e205679ec0a20f33e7 WatchSource:0}: Error finding container e9e9d85e04cf21c42d3e7210cad41ede3ce21f226d4034e205679ec0a20f33e7: Status 404 returned error can't find the container with id e9e9d85e04cf21c42d3e7210cad41ede3ce21f226d4034e205679ec0a20f33e7
Jul 10 15:07:14.677031 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:14.677007    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t" event={"ID":"b5121aaa-a00f-40d5-bd6c-809bac73f938","Type":"ContainerStarted","Data":"e9e9d85e04cf21c42d3e7210cad41ede3ce21f226d4034e205679ec0a20f33e7"}
Jul 10 15:07:15.683087 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:15.683051    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t" event={"ID":"b5121aaa-a00f-40d5-bd6c-809bac73f938","Type":"ContainerStarted","Data":"ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9"}
Jul 10 15:07:15.705108 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:15.705060    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t" podStartSLOduration=1.705046153 podStartE2EDuration="1.705046153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:07:15.702428918 +0000 UTC m=+1933.408750512" watchObservedRunningTime="2026-07-10 15:07:15.705046153 +0000 UTC m=+1933.411367623"
Jul 10 15:07:17.288561 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:17.288508    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:07:17.604853 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:17.604755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:17.757595 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:17.757555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:24.511033 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:24.510995    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:24.511420 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:24.511053    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:24.523595 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:24.523570    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:24.743088 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:24.743058    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:07:25.247980 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:25.247944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:25.352130 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:25.352095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:32.364094 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:32.364054    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:07:39.548752 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:39.548716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:07:43.467850 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:07:43.467813    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:08:01.295751 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:01.295673    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"]
Jul 10 15:08:01.321990 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:01.321944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:02.258724 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:02.258683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:02.460776 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:02.460741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:02.905870 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:02.905837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:03.116315 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.116277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:03.215977 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.215940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:03.389972 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.389942    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"]
Jul 10 15:08:03.390226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.390196    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" podUID="52ce3e86-db61-4f94-8d8d-5143d594264d" containerName="maas-api" containerID="cri-o://9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0" gracePeriod=30
Jul 10 15:08:03.424580 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.424550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:03.652610 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.652584    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:08:03.653569 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.653547    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52ce3e86-db61-4f94-8d8d-5143d594264d-openshift-service-ca\") pod \"52ce3e86-db61-4f94-8d8d-5143d594264d\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") "
Jul 10 15:08:03.653670 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.653584    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls\") pod \"52ce3e86-db61-4f94-8d8d-5143d594264d\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") "
Jul 10 15:08:03.653670 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.653652    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp9t7\" (UniqueName: \"kubernetes.io/projected/52ce3e86-db61-4f94-8d8d-5143d594264d-kube-api-access-hp9t7\") pod \"52ce3e86-db61-4f94-8d8d-5143d594264d\" (UID: \"52ce3e86-db61-4f94-8d8d-5143d594264d\") "
Jul 10 15:08:03.653944 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.653916    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52ce3e86-db61-4f94-8d8d-5143d594264d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52ce3e86-db61-4f94-8d8d-5143d594264d" (UID: "52ce3e86-db61-4f94-8d8d-5143d594264d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:08:03.655947 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.655918    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52ce3e86-db61-4f94-8d8d-5143d594264d-kube-api-access-hp9t7" (OuterVolumeSpecName: "kube-api-access-hp9t7") pod "52ce3e86-db61-4f94-8d8d-5143d594264d" (UID: "52ce3e86-db61-4f94-8d8d-5143d594264d"). InnerVolumeSpecName "kube-api-access-hp9t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:08:03.656074 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.655956    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52ce3e86-db61-4f94-8d8d-5143d594264d" (UID: "52ce3e86-db61-4f94-8d8d-5143d594264d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:08:03.754727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.754692    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52ce3e86-db61-4f94-8d8d-5143d594264d-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:03.754727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.754720    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52ce3e86-db61-4f94-8d8d-5143d594264d-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:03.754727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.754729    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp9t7\" (UniqueName: \"kubernetes.io/projected/52ce3e86-db61-4f94-8d8d-5143d594264d-kube-api-access-hp9t7\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:03.952283 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.952248    2583 generic.go:356] "Generic (PLEG): container finished" podID="52ce3e86-db61-4f94-8d8d-5143d594264d" containerID="9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0" exitCode=0
Jul 10 15:08:03.952461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.952301    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"
Jul 10 15:08:03.952461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.952334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" event={"ID":"52ce3e86-db61-4f94-8d8d-5143d594264d","Type":"ContainerDied","Data":"9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0"}
Jul 10 15:08:03.952461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.952389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" event={"ID":"52ce3e86-db61-4f94-8d8d-5143d594264d","Type":"ContainerDied","Data":"a7b10be75c7f49253e9f6228899a21d0e98803bacb0f167587f314b0b2fb12b0"}
Jul 10 15:08:03.952461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.952425    2583 scope.go:117] "RemoveContainer" containerID="9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0"
Jul 10 15:08:03.963816 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.963557    2583 scope.go:117] "RemoveContainer" containerID="9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0"
Jul 10 15:08:03.964100 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:08:03.964079    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0\": container with ID starting with 9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0 not found: ID does not exist" containerID="9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0"
Jul 10 15:08:03.964173 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.964108    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0"} err="failed to get container status \"9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0\": rpc error: code = NotFound desc = could not find container \"9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0\": container with ID starting with 9750c40718e61537c8c06658d241e9a0087505ef8a7c97153178d8f2b30e91c0 not found: ID does not exist"
Jul 10 15:08:03.992040 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:03.992008    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"]
Jul 10 15:08:04.008456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:04.008428    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch"]
Jul 10 15:08:04.846209 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:04.846177    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52ce3e86-db61-4f94-8d8d-5143d594264d" path="/var/lib/kubelet/pods/52ce3e86-db61-4f94-8d8d-5143d594264d/volumes"
Jul 10 15:08:06.588420 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:06.588360    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e6d308-7779c79977-g55ch" podUID="52ce3e86-db61-4f94-8d8d-5143d594264d" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.59:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 10 15:08:06.780003 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:06.779972    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:08:07.910150 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:07.910117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:09.529477 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:09.529440    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:09.597716 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:09.597682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:16.320245 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.320191    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t" podUID="b5121aaa-a00f-40d5-bd6c-809bac73f938" containerName="main" containerID="cri-o://ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9" gracePeriod=45
Jul 10 15:08:16.572950 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.572892    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:08:16.671386 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671346    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-tmp-dir\") pod \"b5121aaa-a00f-40d5-bd6c-809bac73f938\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") "
Jul 10 15:08:16.671567 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671403    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-model-cache\") pod \"b5121aaa-a00f-40d5-bd6c-809bac73f938\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") "
Jul 10 15:08:16.671567 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671436    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-dshm\") pod \"b5121aaa-a00f-40d5-bd6c-809bac73f938\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") "
Jul 10 15:08:16.671567 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671491    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5121aaa-a00f-40d5-bd6c-809bac73f938-tls-certs\") pod \"b5121aaa-a00f-40d5-bd6c-809bac73f938\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") "
Jul 10 15:08:16.671738 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671579    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zjk9\" (UniqueName: \"kubernetes.io/projected/b5121aaa-a00f-40d5-bd6c-809bac73f938-kube-api-access-7zjk9\") pod \"b5121aaa-a00f-40d5-bd6c-809bac73f938\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") "
Jul 10 15:08:16.671738 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671598    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-home\") pod \"b5121aaa-a00f-40d5-bd6c-809bac73f938\" (UID: \"b5121aaa-a00f-40d5-bd6c-809bac73f938\") "
Jul 10 15:08:16.671738 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671678    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5121aaa-a00f-40d5-bd6c-809bac73f938" (UID: "b5121aaa-a00f-40d5-bd6c-809bac73f938"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:08:16.671738 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671700    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5121aaa-a00f-40d5-bd6c-809bac73f938" (UID: "b5121aaa-a00f-40d5-bd6c-809bac73f938"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:08:16.671932 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671878    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-tmp-dir\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:16.671932 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671898    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-model-cache\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:16.671998 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.671930    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-home" (OuterVolumeSpecName: "home") pod "b5121aaa-a00f-40d5-bd6c-809bac73f938" (UID: "b5121aaa-a00f-40d5-bd6c-809bac73f938"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:08:16.673684 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.673653    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-dshm" (OuterVolumeSpecName: "dshm") pod "b5121aaa-a00f-40d5-bd6c-809bac73f938" (UID: "b5121aaa-a00f-40d5-bd6c-809bac73f938"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:08:16.673787 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.673723    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5121aaa-a00f-40d5-bd6c-809bac73f938-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5121aaa-a00f-40d5-bd6c-809bac73f938" (UID: "b5121aaa-a00f-40d5-bd6c-809bac73f938"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:08:16.673787 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.673735    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5121aaa-a00f-40d5-bd6c-809bac73f938-kube-api-access-7zjk9" (OuterVolumeSpecName: "kube-api-access-7zjk9") pod "b5121aaa-a00f-40d5-bd6c-809bac73f938" (UID: "b5121aaa-a00f-40d5-bd6c-809bac73f938"). InnerVolumeSpecName "kube-api-access-7zjk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:08:16.773180 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.773152    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-dshm\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:16.773180 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.773178    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5121aaa-a00f-40d5-bd6c-809bac73f938-tls-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:16.773368 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.773189    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zjk9\" (UniqueName: \"kubernetes.io/projected/b5121aaa-a00f-40d5-bd6c-809bac73f938-kube-api-access-7zjk9\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:16.773368 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:16.773198    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5121aaa-a00f-40d5-bd6c-809bac73f938-home\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:17.020735 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.020699    2583 generic.go:356] "Generic (PLEG): container finished" podID="b5121aaa-a00f-40d5-bd6c-809bac73f938" containerID="ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9" exitCode=0
Jul 10 15:08:17.020905 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.020747    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"
Jul 10 15:08:17.020905 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.020785    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t" event={"ID":"b5121aaa-a00f-40d5-bd6c-809bac73f938","Type":"ContainerDied","Data":"ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9"}
Jul 10 15:08:17.020905 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.020833    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t" event={"ID":"b5121aaa-a00f-40d5-bd6c-809bac73f938","Type":"ContainerDied","Data":"e9e9d85e04cf21c42d3e7210cad41ede3ce21f226d4034e205679ec0a20f33e7"}
Jul 10 15:08:17.020905 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.020853    2583 scope.go:117] "RemoveContainer" containerID="ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9"
Jul 10 15:08:17.029825 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.029805    2583 scope.go:117] "RemoveContainer" containerID="ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9"
Jul 10 15:08:17.030087 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:08:17.030055    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9\": container with ID starting with ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9 not found: ID does not exist" containerID="ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9"
Jul 10 15:08:17.030130 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.030084    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9"} err="failed to get container status \"ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9\": rpc error: code = NotFound desc = could not find container \"ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9\": container with ID starting with ec5d42e77dabe44cddeef44912d1bc6b913ca6c4f683415e5a588c9c4c8947a9 not found: ID does not exist"
Jul 10 15:08:17.045542 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.045502    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"]
Jul 10 15:08:17.053990 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:17.053970    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e6d308/test-model-e6d308-kserve-68dc5f49f7-xhf8t"]
Jul 10 15:08:18.847234 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:18.847198    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5121aaa-a00f-40d5-bd6c-809bac73f938" path="/var/lib/kubelet/pods/b5121aaa-a00f-40d5-bd6c-809bac73f938/volumes"
Jul 10 15:08:45.418496 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:45.418455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:45.708188 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:45.708105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:45.905683 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:45.905647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:46.203868 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:46.203835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:46.574453 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:46.574368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:46.746812 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:46.746778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:46.839117 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:46.839032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:47.004496 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.004455    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"]
Jul 10 15:08:47.004756 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.004724    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" podUID="78eff4d6-3404-4310-89f4-9ac319d66161" containerName="maas-api" containerID="cri-o://92f729015bd5005f4361d95140f4b5b3a32d602d8c9826219ee5a4c09c21cb0c" gracePeriod=30
Jul 10 15:08:47.041976 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.041946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:47.178833 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.178800    2583 generic.go:356] "Generic (PLEG): container finished" podID="78eff4d6-3404-4310-89f4-9ac319d66161" containerID="92f729015bd5005f4361d95140f4b5b3a32d602d8c9826219ee5a4c09c21cb0c" exitCode=0
Jul 10 15:08:47.178987 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.178875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" event={"ID":"78eff4d6-3404-4310-89f4-9ac319d66161","Type":"ContainerDied","Data":"92f729015bd5005f4361d95140f4b5b3a32d602d8c9826219ee5a4c09c21cb0c"}
Jul 10 15:08:47.261651 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.261629    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:08:47.359483 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.359452    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls\") pod \"78eff4d6-3404-4310-89f4-9ac319d66161\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") "
Jul 10 15:08:47.359671 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.359496    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjqmv\" (UniqueName: \"kubernetes.io/projected/78eff4d6-3404-4310-89f4-9ac319d66161-kube-api-access-cjqmv\") pod \"78eff4d6-3404-4310-89f4-9ac319d66161\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") "
Jul 10 15:08:47.359671 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.359626    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eff4d6-3404-4310-89f4-9ac319d66161-openshift-service-ca\") pod \"78eff4d6-3404-4310-89f4-9ac319d66161\" (UID: \"78eff4d6-3404-4310-89f4-9ac319d66161\") "
Jul 10 15:08:47.359969 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.359943    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78eff4d6-3404-4310-89f4-9ac319d66161-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78eff4d6-3404-4310-89f4-9ac319d66161" (UID: "78eff4d6-3404-4310-89f4-9ac319d66161"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:08:47.361613 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.361582    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78eff4d6-3404-4310-89f4-9ac319d66161" (UID: "78eff4d6-3404-4310-89f4-9ac319d66161"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:08:47.361613 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.361590    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78eff4d6-3404-4310-89f4-9ac319d66161-kube-api-access-cjqmv" (OuterVolumeSpecName: "kube-api-access-cjqmv") pod "78eff4d6-3404-4310-89f4-9ac319d66161" (UID: "78eff4d6-3404-4310-89f4-9ac319d66161"). InnerVolumeSpecName "kube-api-access-cjqmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:08:47.460316 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.460289    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eff4d6-3404-4310-89f4-9ac319d66161-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:47.460316 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.460316    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78eff4d6-3404-4310-89f4-9ac319d66161-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:47.460486 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:47.460326    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjqmv\" (UniqueName: \"kubernetes.io/projected/78eff4d6-3404-4310-89f4-9ac319d66161-kube-api-access-cjqmv\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:08:48.185725 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:48.185693    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"
Jul 10 15:08:48.186127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:48.185741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l" event={"ID":"78eff4d6-3404-4310-89f4-9ac319d66161","Type":"ContainerDied","Data":"f5dbf4609218bf6b7f5b9b0afa349ff3d475252f39c556e17e0d1f70fe8be72e"}
Jul 10 15:08:48.186127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:48.185798    2583 scope.go:117] "RemoveContainer" containerID="92f729015bd5005f4361d95140f4b5b3a32d602d8c9826219ee5a4c09c21cb0c"
Jul 10 15:08:48.209355 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:48.209331    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"]
Jul 10 15:08:48.215720 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:48.215700    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff844d-7f66468c6b-ws94l"]
Jul 10 15:08:48.847774 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:48.847742    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78eff4d6-3404-4310-89f4-9ac319d66161" path="/var/lib/kubelet/pods/78eff4d6-3404-4310-89f4-9ac319d66161/volumes"
Jul 10 15:08:49.978371 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:49.978336    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:08:53.428808 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:53.428770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:53.528116 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:53.528081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:53.727604 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:53.727490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:53.929069 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:53.929024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:54.269813 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:54.269782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:54.581350 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:54.581256    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:54.867073 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:54.866990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:57.151300 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.151265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:57.247433 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247401    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"]
Jul 10 15:08:57.247852 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247837    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5121aaa-a00f-40d5-bd6c-809bac73f938" containerName="main"
Jul 10 15:08:57.247903 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247853    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5121aaa-a00f-40d5-bd6c-809bac73f938" containerName="main"
Jul 10 15:08:57.247903 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247874    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78eff4d6-3404-4310-89f4-9ac319d66161" containerName="maas-api"
Jul 10 15:08:57.247903 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247880    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="78eff4d6-3404-4310-89f4-9ac319d66161" containerName="maas-api"
Jul 10 15:08:57.247903 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247888    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52ce3e86-db61-4f94-8d8d-5143d594264d" containerName="maas-api"
Jul 10 15:08:57.247903 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247895    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="52ce3e86-db61-4f94-8d8d-5143d594264d" containerName="maas-api"
Jul 10 15:08:57.248051 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247956    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="52ce3e86-db61-4f94-8d8d-5143d594264d" containerName="maas-api"
Jul 10 15:08:57.248051 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247967    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5121aaa-a00f-40d5-bd6c-809bac73f938" containerName="main"
Jul 10 15:08:57.248051 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.247976    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="78eff4d6-3404-4310-89f4-9ac319d66161" containerName="maas-api"
Jul 10 15:08:57.252605 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.252580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.256348 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.256320    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6847a8-serving-cert\""
Jul 10 15:08:57.265681 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.265655    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"]
Jul 10 15:08:57.267837 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.267812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:57.344947 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.344906    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f323af-618e-4029-a18b-5e1f3a5a864a-openshift-service-ca\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.345127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.344958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/66f323af-618e-4029-a18b-5e1f3a5a864a-kube-api-access-948qw\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.345127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.345026    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.369691 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.369648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:57.445504 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.445472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f323af-618e-4029-a18b-5e1f3a5a864a-openshift-service-ca\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.445671 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.445613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/66f323af-618e-4029-a18b-5e1f3a5a864a-kube-api-access-948qw\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.445671 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.445654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.445831 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:08:57.445781    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-serving-cert: secret "maas-api-e2e-api-a-6847a8-serving-cert" not found
Jul 10 15:08:57.445913 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:08:57.445860    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls podName:66f323af-618e-4029-a18b-5e1f3a5a864a nodeName:}" failed. No retries permitted until 2026-07-10 15:08:57.945839003 +0000 UTC m=+2035.652160471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls") pod "maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" (UID: "66f323af-618e-4029-a18b-5e1f3a5a864a") : secret "maas-api-e2e-api-a-6847a8-serving-cert" not found
Jul 10 15:08:57.446190 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.446165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f323af-618e-4029-a18b-5e1f3a5a864a-openshift-service-ca\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.457541 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.457499    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/66f323af-618e-4029-a18b-5e1f3a5a864a-kube-api-access-948qw\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.913188 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.913151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:57.951570 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.951495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:57.953962 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:57.953936    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls\") pod \"maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:58.011360 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:58.011312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:08:58.165147 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:58.165057    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:58.331029 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:58.330982    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"]
Jul 10 15:08:58.336295 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:08:58.336249    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f323af_618e_4029_a18b_5e1f3a5a864a.slice/crio-9e0e76e6cd30b7537c5930ce03da978e1d01f6bcd9a4f31da3c5b8ad50503888 WatchSource:0}: Error finding container 9e0e76e6cd30b7537c5930ce03da978e1d01f6bcd9a4f31da3c5b8ad50503888: Status 404 returned error can't find the container with id 9e0e76e6cd30b7537c5930ce03da978e1d01f6bcd9a4f31da3c5b8ad50503888
Jul 10 15:08:59.248894 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:59.248853    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" event={"ID":"66f323af-618e-4029-a18b-5e1f3a5a864a","Type":"ContainerStarted","Data":"4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77"}
Jul 10 15:08:59.249355 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:59.248950    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" event={"ID":"66f323af-618e-4029-a18b-5e1f3a5a864a","Type":"ContainerStarted","Data":"9e0e76e6cd30b7537c5930ce03da978e1d01f6bcd9a4f31da3c5b8ad50503888"}
Jul 10 15:08:59.249355 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:59.249083    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:08:59.378689 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:08:59.378638    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" podStartSLOduration=1.967424032 podStartE2EDuration="2.378620183s" totalImagesPullingTime="411.196151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:08:59.37101382 +0000 UTC m=+2037.077335289" watchObservedRunningTime="2026-07-10 15:08:59.378620183 +0000 UTC m=+2037.084941651"
Jul 10 15:09:01.621868 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:01.621831    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:02.069129 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:02.069096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:02.401348 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:02.401265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:02.775722 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:02.775687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:04.002714 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:04.002683    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:09:05.260083 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.260052    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:09:05.301662 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.301619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:05.396396 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.396363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:05.406225 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.406192    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"]
Jul 10 15:09:05.412379 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.412350    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.417808 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.417779    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1715f8-serving-cert\""
Jul 10 15:09:05.423368 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.423331    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"]
Jul 10 15:09:05.491776 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.491733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:05.525956 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.525869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e986f806-4b2a-4d20-9856-38ad792d576c-openshift-service-ca\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.526111 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.525977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwk6b\" (UniqueName: \"kubernetes.io/projected/e986f806-4b2a-4d20-9856-38ad792d576c-kube-api-access-gwk6b\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.526111 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.526042    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e986f806-4b2a-4d20-9856-38ad792d576c-maas-api-tls\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.626974 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.626932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwk6b\" (UniqueName: \"kubernetes.io/projected/e986f806-4b2a-4d20-9856-38ad792d576c-kube-api-access-gwk6b\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.627153 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.626998    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e986f806-4b2a-4d20-9856-38ad792d576c-maas-api-tls\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.627153 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.627039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e986f806-4b2a-4d20-9856-38ad792d576c-openshift-service-ca\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.627719 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.627690    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e986f806-4b2a-4d20-9856-38ad792d576c-openshift-service-ca\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.629335 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.629315    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e986f806-4b2a-4d20-9856-38ad792d576c-maas-api-tls\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.634436 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.634402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:05.636226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.636193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwk6b\" (UniqueName: \"kubernetes.io/projected/e986f806-4b2a-4d20-9856-38ad792d576c-kube-api-access-gwk6b\") pod \"maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.725629 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.725596    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:05.871505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:05.871479    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"]
Jul 10 15:09:05.873924 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:09:05.873898    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode986f806_4b2a_4d20_9856_38ad792d576c.slice/crio-2af12d515ffa5354ce4b266eee0b83b1f1ce965e5addf84127cfb419793c4621 WatchSource:0}: Error finding container 2af12d515ffa5354ce4b266eee0b83b1f1ce965e5addf84127cfb419793c4621: Status 404 returned error can't find the container with id 2af12d515ffa5354ce4b266eee0b83b1f1ce965e5addf84127cfb419793c4621
Jul 10 15:09:06.024136 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:06.024102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:06.125259 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:06.125160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:06.290438 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:06.290400    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56" event={"ID":"e986f806-4b2a-4d20-9856-38ad792d576c","Type":"ContainerStarted","Data":"0ff58c37a2f9754a5b287c3b5cdfefcba989c3c4ef52a8d77e2e2008833d0932"}
Jul 10 15:09:06.290852 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:06.290560    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:06.290852 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:06.290597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56" event={"ID":"e986f806-4b2a-4d20-9856-38ad792d576c","Type":"ContainerStarted","Data":"2af12d515ffa5354ce4b266eee0b83b1f1ce965e5addf84127cfb419793c4621"}
Jul 10 15:09:06.312337 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:06.312269    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56" podStartSLOduration=1.000144079 podStartE2EDuration="1.312251087s" totalImagesPullingTime="312.107008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:09:06.311820943 +0000 UTC m=+2044.018142413" watchObservedRunningTime="2026-07-10 15:09:06.312251087 +0000 UTC m=+2044.018572547"
Jul 10 15:09:08.268902 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:08.268871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:08.370186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:08.370155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:09.186148 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:09.186115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:09.283710 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:09.283665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:09.680875 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:09.680842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:12.301879 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:12.301842    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:17.803191 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:17.803153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:18.144293 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.144203    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:18.294941 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.294903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:18.394195 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.394164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:18.562770 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.562732    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"]
Jul 10 15:09:18.563002 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.562974    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" podUID="66f323af-618e-4029-a18b-5e1f3a5a864a" containerName="maas-api" containerID="cri-o://4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77" gracePeriod=30
Jul 10 15:09:18.592932 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.592900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:18.844339 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.844313    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:09:18.957068 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.957032    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls\") pod \"66f323af-618e-4029-a18b-5e1f3a5a864a\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") "
Jul 10 15:09:18.957229 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.957076    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/66f323af-618e-4029-a18b-5e1f3a5a864a-kube-api-access-948qw\") pod \"66f323af-618e-4029-a18b-5e1f3a5a864a\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") "
Jul 10 15:09:18.957229 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.957139    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f323af-618e-4029-a18b-5e1f3a5a864a-openshift-service-ca\") pod \"66f323af-618e-4029-a18b-5e1f3a5a864a\" (UID: \"66f323af-618e-4029-a18b-5e1f3a5a864a\") "
Jul 10 15:09:18.957622 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.957574    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f323af-618e-4029-a18b-5e1f3a5a864a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66f323af-618e-4029-a18b-5e1f3a5a864a" (UID: "66f323af-618e-4029-a18b-5e1f3a5a864a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:09:18.959253 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.959230    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66f323af-618e-4029-a18b-5e1f3a5a864a" (UID: "66f323af-618e-4029-a18b-5e1f3a5a864a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:09:18.959383 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:18.959301    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f323af-618e-4029-a18b-5e1f3a5a864a-kube-api-access-948qw" (OuterVolumeSpecName: "kube-api-access-948qw") pod "66f323af-618e-4029-a18b-5e1f3a5a864a" (UID: "66f323af-618e-4029-a18b-5e1f3a5a864a"). InnerVolumeSpecName "kube-api-access-948qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:09:19.058924 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.058887    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66f323af-618e-4029-a18b-5e1f3a5a864a-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:09:19.058924 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.058921    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-948qw\" (UniqueName: \"kubernetes.io/projected/66f323af-618e-4029-a18b-5e1f3a5a864a-kube-api-access-948qw\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:09:19.059162 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.058937    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f323af-618e-4029-a18b-5e1f3a5a864a-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:09:19.365542 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.365481    2583 generic.go:356] "Generic (PLEG): container finished" podID="66f323af-618e-4029-a18b-5e1f3a5a864a" containerID="4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77" exitCode=0
Jul 10 15:09:19.365542 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.365540    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"
Jul 10 15:09:19.365800 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.365540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" event={"ID":"66f323af-618e-4029-a18b-5e1f3a5a864a","Type":"ContainerDied","Data":"4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77"}
Jul 10 15:09:19.365800 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.365607    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr" event={"ID":"66f323af-618e-4029-a18b-5e1f3a5a864a","Type":"ContainerDied","Data":"9e0e76e6cd30b7537c5930ce03da978e1d01f6bcd9a4f31da3c5b8ad50503888"}
Jul 10 15:09:19.365800 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.365637    2583 scope.go:117] "RemoveContainer" containerID="4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77"
Jul 10 15:09:19.375713 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.375697    2583 scope.go:117] "RemoveContainer" containerID="4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77"
Jul 10 15:09:19.375967 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:09:19.375952    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77\": container with ID starting with 4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77 not found: ID does not exist" containerID="4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77"
Jul 10 15:09:19.376010 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.375975    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77"} err="failed to get container status \"4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77\": rpc error: code = NotFound desc = could not find container \"4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77\": container with ID starting with 4b331be5f6e202129548e7064ec1877c3d9b048a8fd5418abf9088b4ebc91f77 not found: ID does not exist"
Jul 10 15:09:19.390211 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.390179    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"]
Jul 10 15:09:19.395554 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:19.395532    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6847a8-77c7684f7d-xp5fr"]
Jul 10 15:09:20.846565 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:20.846505    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f323af-618e-4029-a18b-5e1f3a5a864a" path="/var/lib/kubelet/pods/66f323af-618e-4029-a18b-5e1f3a5a864a/volumes"
Jul 10 15:09:22.271151 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:22.271070    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:09:24.760085 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:24.759994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:24.897449 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:24.897415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:25.026732 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:25.026628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:25.241334 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:25.241297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:25.418173 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:25.418137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:25.802052 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:25.801955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:25.933935 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:25.933893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:26.036090 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.036057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:26.216491 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.216456    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"]
Jul 10 15:09:26.216750 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.216723    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56" podUID="e986f806-4b2a-4d20-9856-38ad792d576c" containerName="maas-api" containerID="cri-o://0ff58c37a2f9754a5b287c3b5cdfefcba989c3c4ef52a8d77e2e2008833d0932" gracePeriod=30
Jul 10 15:09:26.248799 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.248773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:26.410409 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.410373    2583 generic.go:356] "Generic (PLEG): container finished" podID="e986f806-4b2a-4d20-9856-38ad792d576c" containerID="0ff58c37a2f9754a5b287c3b5cdfefcba989c3c4ef52a8d77e2e2008833d0932" exitCode=0
Jul 10 15:09:26.410607 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.410437    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56" event={"ID":"e986f806-4b2a-4d20-9856-38ad792d576c","Type":"ContainerDied","Data":"0ff58c37a2f9754a5b287c3b5cdfefcba989c3c4ef52a8d77e2e2008833d0932"}
Jul 10 15:09:26.470685 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.470631    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:26.628712 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.628679    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e986f806-4b2a-4d20-9856-38ad792d576c-openshift-service-ca\") pod \"e986f806-4b2a-4d20-9856-38ad792d576c\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") "
Jul 10 15:09:26.628898 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.628757    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e986f806-4b2a-4d20-9856-38ad792d576c-maas-api-tls\") pod \"e986f806-4b2a-4d20-9856-38ad792d576c\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") "
Jul 10 15:09:26.628898 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.628807    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwk6b\" (UniqueName: \"kubernetes.io/projected/e986f806-4b2a-4d20-9856-38ad792d576c-kube-api-access-gwk6b\") pod \"e986f806-4b2a-4d20-9856-38ad792d576c\" (UID: \"e986f806-4b2a-4d20-9856-38ad792d576c\") "
Jul 10 15:09:26.629024 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.628986    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e986f806-4b2a-4d20-9856-38ad792d576c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e986f806-4b2a-4d20-9856-38ad792d576c" (UID: "e986f806-4b2a-4d20-9856-38ad792d576c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:09:26.630918 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.630889    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e986f806-4b2a-4d20-9856-38ad792d576c-kube-api-access-gwk6b" (OuterVolumeSpecName: "kube-api-access-gwk6b") pod "e986f806-4b2a-4d20-9856-38ad792d576c" (UID: "e986f806-4b2a-4d20-9856-38ad792d576c"). InnerVolumeSpecName "kube-api-access-gwk6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:09:26.631036 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.630921    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e986f806-4b2a-4d20-9856-38ad792d576c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e986f806-4b2a-4d20-9856-38ad792d576c" (UID: "e986f806-4b2a-4d20-9856-38ad792d576c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:09:26.729505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.729427    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e986f806-4b2a-4d20-9856-38ad792d576c-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:09:26.729505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.729458    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e986f806-4b2a-4d20-9856-38ad792d576c-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:09:26.729505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:26.729468    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwk6b\" (UniqueName: \"kubernetes.io/projected/e986f806-4b2a-4d20-9856-38ad792d576c-kube-api-access-gwk6b\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:09:27.416459 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:27.416431    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"
Jul 10 15:09:27.416956 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:27.416458    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56" event={"ID":"e986f806-4b2a-4d20-9856-38ad792d576c","Type":"ContainerDied","Data":"2af12d515ffa5354ce4b266eee0b83b1f1ce965e5addf84127cfb419793c4621"}
Jul 10 15:09:27.416956 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:27.416559    2583 scope.go:117] "RemoveContainer" containerID="0ff58c37a2f9754a5b287c3b5cdfefcba989c3c4ef52a8d77e2e2008833d0932"
Jul 10 15:09:27.438130 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:27.438076    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"]
Jul 10 15:09:27.440434 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:27.440412    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1715f8-67fdcbc645-9lp56"]
Jul 10 15:09:28.846872 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:28.846836    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e986f806-4b2a-4d20-9856-38ad792d576c" path="/var/lib/kubelet/pods/e986f806-4b2a-4d20-9856-38ad792d576c/volumes"
Jul 10 15:09:32.718378 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:32.718333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:32.788221 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:32.788190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:33.010626 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:33.010509    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:33.243836 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:33.243791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:33.558461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:33.558411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:33.887091 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:33.887013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:34.174752 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:34.174720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:35.679006 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:35.678976    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:09:36.426019 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.425983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:36.534138 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534103    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"]
Jul 10 15:09:36.534755 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534733    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f323af-618e-4029-a18b-5e1f3a5a864a" containerName="maas-api"
Jul 10 15:09:36.534755 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534755    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f323af-618e-4029-a18b-5e1f3a5a864a" containerName="maas-api"
Jul 10 15:09:36.535001 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534786    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e986f806-4b2a-4d20-9856-38ad792d576c" containerName="maas-api"
Jul 10 15:09:36.535001 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534795    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e986f806-4b2a-4d20-9856-38ad792d576c" containerName="maas-api"
Jul 10 15:09:36.535001 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534880    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e986f806-4b2a-4d20-9856-38ad792d576c" containerName="maas-api"
Jul 10 15:09:36.535001 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.534894    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f323af-618e-4029-a18b-5e1f3a5a864a" containerName="maas-api"
Jul 10 15:09:36.535299 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.535147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:36.537948 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.537932    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.543605 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.543587    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-7b13c8-serving-cert\""
Jul 10 15:09:36.554370 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.554347    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"]
Jul 10 15:09:36.608927 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.608900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.609073 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.608932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h86fz\" (UniqueName: \"kubernetes.io/projected/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-kube-api-access-h86fz\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.609073 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.608973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.634455 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.634411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:36.709653 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.709582    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.709653 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.709616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h86fz\" (UniqueName: \"kubernetes.io/projected/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-kube-api-access-h86fz\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.710028 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.709665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.710028 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:09:36.709717    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-serving-cert: secret "maas-api-e2e-shared-a-7b13c8-serving-cert" not found
Jul 10 15:09:36.710028 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:09:36.709783    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls podName:ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3 nodeName:}" failed. No retries permitted until 2026-07-10 15:09:37.209761621 +0000 UTC m=+2074.916083089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls") pod "maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" (UID: "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3") : secret "maas-api-e2e-shared-a-7b13c8-serving-cert" not found
Jul 10 15:09:36.710333 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.710315    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.719382 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.719349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h86fz\" (UniqueName: \"kubernetes.io/projected/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-kube-api-access-h86fz\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:36.874895 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:36.874861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:37.214996 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:37.214965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:37.217288 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:37.217259    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:37.277796 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:37.277761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:37.392411 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:37.392379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:37.448975 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:37.448936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:37.589598 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:37.589571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"]
Jul 10 15:09:37.591844 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:09:37.591810    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9859c6_94a9_4e4a_bb0d_f5c6d2e171f3.slice/crio-2a760c57002f3f05d61cffc6bdf46e765391025e20f11021e5bb60a10b7b7704 WatchSource:0}: Error finding container 2a760c57002f3f05d61cffc6bdf46e765391025e20f11021e5bb60a10b7b7704: Status 404 returned error can't find the container with id 2a760c57002f3f05d61cffc6bdf46e765391025e20f11021e5bb60a10b7b7704
Jul 10 15:09:38.483130 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:38.483099    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" event={"ID":"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3","Type":"ContainerStarted","Data":"8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba"}
Jul 10 15:09:38.483507 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:38.483155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" event={"ID":"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3","Type":"ContainerStarted","Data":"2a760c57002f3f05d61cffc6bdf46e765391025e20f11021e5bb60a10b7b7704"}
Jul 10 15:09:38.483507 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:38.483180    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:38.514877 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:38.514832    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" podStartSLOduration=2.117169195 podStartE2EDuration="2.514817082s" totalImagesPullingTime="397.647887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:09:38.51319903 +0000 UTC m=+2076.219520511" watchObservedRunningTime="2026-07-10 15:09:38.514817082 +0000 UTC m=+2076.221138770"
Jul 10 15:09:40.914752 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:40.914715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:41.236666 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:41.236570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:41.752368 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:41.752332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:41.991779 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:41.991742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:44.493698 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.493667    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:09:44.517771 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.517734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:44.642477 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.642443    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"]
Jul 10 15:09:44.646220 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.646204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.651820 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.651789    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e538f1-serving-cert\""
Jul 10 15:09:44.666840 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.666817    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"]
Jul 10 15:09:44.672311 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.672273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:44.684393 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.684370    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnhbl\" (UniqueName: \"kubernetes.io/projected/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-kube-api-access-vnhbl\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.684504 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.684484    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-maas-api-tls\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.684634 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.684609    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.769859 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.769795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:44.785892 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.785848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.785892 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.785882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnhbl\" (UniqueName: \"kubernetes.io/projected/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-kube-api-access-vnhbl\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.786022 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.785933    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-maas-api-tls\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.786488 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.786469    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.788243 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.788222    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-maas-api-tls\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.798910 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.798889    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnhbl\" (UniqueName: \"kubernetes.io/projected/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-kube-api-access-vnhbl\") pod \"maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:44.867938 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.867904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:44.956382 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:44.956356    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:45.099109 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:45.099081    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"]
Jul 10 15:09:45.101650 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:09:45.101620    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3fdcd28_f6d0_4ea4_98fc_d139714ea282.slice/crio-bc7030183f391cd083065357fb4e1ba836f9266f81f1a89e32077519f9589e46 WatchSource:0}: Error finding container bc7030183f391cd083065357fb4e1ba836f9266f81f1a89e32077519f9589e46: Status 404 returned error can't find the container with id bc7030183f391cd083065357fb4e1ba836f9266f81f1a89e32077519f9589e46
Jul 10 15:09:45.308799 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:45.308712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:45.483456 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:45.483425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:45.524842 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:45.524804    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9" event={"ID":"b3fdcd28-f6d0-4ea4-98fc-d139714ea282","Type":"ContainerStarted","Data":"bc7030183f391cd083065357fb4e1ba836f9266f81f1a89e32077519f9589e46"}
Jul 10 15:09:46.531419 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:46.531385    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9" event={"ID":"b3fdcd28-f6d0-4ea4-98fc-d139714ea282","Type":"ContainerStarted","Data":"5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b"}
Jul 10 15:09:46.531852 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:46.531500    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:46.555660 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:46.555611    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9" podStartSLOduration=2.191217136 podStartE2EDuration="2.555597712s" totalImagesPullingTime="364.380576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:09:46.553241248 +0000 UTC m=+2084.259562717" watchObservedRunningTime="2026-07-10 15:09:46.555597712 +0000 UTC m=+2084.261919181"
Jul 10 15:09:48.170052 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:48.170012    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:09:49.016302 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:49.016262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:50.390665 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:50.390631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:52.542662 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:52.542630    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:09:58.190677 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:58.190642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:59.743133 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:59.743094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:09:59.839368 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:09:59.839329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:10:03.114541 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:03.114484    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:10:03.162428 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:03.162398    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:10:03.167946 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:03.167925    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:10:06.323549 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:06.323493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:10:06.422145 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:06.422104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:10:07.310466 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:07.310424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:10:14.612002 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.611965    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"]
Jul 10 15:10:14.615687 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.615663    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.619953 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.619927    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"auth-test-model-e538f1-kserve-self-signed-certs\""
Jul 10 15:10:14.620081 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.619955    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"openshift-service-ca.crt\""
Jul 10 15:10:14.620081 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.620060    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"kube-root-ca.crt\""
Jul 10 15:10:14.620216 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.620198    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"default-dockercfg-kczhc\""
Jul 10 15:10:14.634801 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.634768    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"]
Jul 10 15:10:14.655717 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.655675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:10:14.664846 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.664812    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tmp-dir\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.665007 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.664852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tls-certs\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.665007 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.664934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2jcw\" (UniqueName: \"kubernetes.io/projected/04e09cb4-15e4-44e7-a5c3-205ec665ca95-kube-api-access-c2jcw\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.665007 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.664962    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-home\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.665130 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.665014    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-model-cache\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.665130 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.665096    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-dshm\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.765944 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.765903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tmp-dir\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.765944 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.765943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tls-certs\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.765996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2jcw\" (UniqueName: \"kubernetes.io/projected/04e09cb4-15e4-44e7-a5c3-205ec665ca95-kube-api-access-c2jcw\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.766015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-home\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766186 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.766053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-model-cache\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766352 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.766179    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-dshm\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766436 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.766412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tmp-dir\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.766445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-home\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.766503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.766466    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-model-cache\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.768333 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.768311    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-dshm\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.768727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.768709    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tls-certs\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.775080 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.775057    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2jcw\" (UniqueName: \"kubernetes.io/projected/04e09cb4-15e4-44e7-a5c3-205ec665ca95-kube-api-access-c2jcw\") pod \"auth-test-model-e538f1-kserve-789f7cd47f-m9pj6\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") " pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:14.925259 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:14.925219    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:15.070396 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:15.070365    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"]
Jul 10 15:10:15.072024 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:10:15.071995    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04e09cb4_15e4_44e7_a5c3_205ec665ca95.slice/crio-6cbd5bf6994b9a2131f498cce3f3ad3bae3d2853fe97792177be835070f45484 WatchSource:0}: Error finding container 6cbd5bf6994b9a2131f498cce3f3ad3bae3d2853fe97792177be835070f45484: Status 404 returned error can't find the container with id 6cbd5bf6994b9a2131f498cce3f3ad3bae3d2853fe97792177be835070f45484
Jul 10 15:10:15.687182 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:15.687149    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6" event={"ID":"04e09cb4-15e4-44e7-a5c3-205ec665ca95","Type":"ContainerStarted","Data":"c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2"}
Jul 10 15:10:15.687591 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:15.687192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6" event={"ID":"04e09cb4-15e4-44e7-a5c3-205ec665ca95","Type":"ContainerStarted","Data":"6cbd5bf6994b9a2131f498cce3f3ad3bae3d2853fe97792177be835070f45484"}
Jul 10 15:10:15.708699 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:15.708649    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6" podStartSLOduration=1.708633413 podStartE2EDuration="1.708633413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:10:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:10:15.707765059 +0000 UTC m=+2113.414086541" watchObservedRunningTime="2026-07-10 15:10:15.708633413 +0000 UTC m=+2113.414954860"
Jul 10 15:10:18.067899 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:18.067863    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:10:24.925558 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:24.925494    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:24.927894 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:24.925577    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:24.938279 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:24.938247    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:25.759008 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:25.758980    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:10:45.681135 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:45.681095    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:10:57.600342 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:10:57.600262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:11:13.571511 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:11:13.571472    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:11:29.374709 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:11:29.374673    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:11:47.868843 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:11:47.868807    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:12:01.072133 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:01.072103    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:12:15.688742 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:15.688688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:12:16.510511 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:16.510476    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"]
Jul 10 15:12:17.396842 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:17.396803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:12:19.677585 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:19.677542    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:12:25.916838 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.916809    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"]
Jul 10 15:12:25.920536 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.920503    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:25.923620 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.923598    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"default-dockercfg-9mkjk\""
Jul 10 15:12:25.923739 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.923603    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"kube-root-ca.crt\""
Jul 10 15:12:25.924611 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.924595    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"sub-test-model-7b13c8-kserve-self-signed-certs\""
Jul 10 15:12:25.924681 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.924642    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"openshift-service-ca.crt\""
Jul 10 15:12:25.934279 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.934211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:12:25.934545 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.934511    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"]
Jul 10 15:12:25.968219 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.968191    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-model-cache\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:25.968344 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.968223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lt5d\" (UniqueName: \"kubernetes.io/projected/4a73c00a-d7e1-41d3-94c7-273e337b892c-kube-api-access-7lt5d\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:25.968344 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.968282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-dshm\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:25.968344 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.968297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-tmp-dir\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:25.968505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.968362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a73c00a-d7e1-41d3-94c7-273e337b892c-tls-certs\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:25.968505 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:25.968404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-home\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.069784 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.069740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-model-cache\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.069784 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.069784    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lt5d\" (UniqueName: \"kubernetes.io/projected/4a73c00a-d7e1-41d3-94c7-273e337b892c-kube-api-access-7lt5d\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070019 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.069837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-dshm\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070019 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.069851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-tmp-dir\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070019 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.069878    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a73c00a-d7e1-41d3-94c7-273e337b892c-tls-certs\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070019 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.069900    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-home\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070246 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.070181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-model-cache\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070246 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.070199    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-tmp-dir\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.070371 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.070349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-home\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.072156 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.072132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-dshm\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.072315 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.072295    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a73c00a-d7e1-41d3-94c7-273e337b892c-tls-certs\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.078956 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.078931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lt5d\" (UniqueName: \"kubernetes.io/projected/4a73c00a-d7e1-41d3-94c7-273e337b892c-kube-api-access-7lt5d\") pod \"sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") " pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.229946 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.229651    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:26.388951 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:26.388923    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"]
Jul 10 15:12:26.392453 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:12:26.392429    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a73c00a_d7e1_41d3_94c7_273e337b892c.slice/crio-b8d1663a85e8e4d5da8c6755381d3b5a52ec1efbbf153b5783d0a5914c4b42ed WatchSource:0}: Error finding container b8d1663a85e8e4d5da8c6755381d3b5a52ec1efbbf153b5783d0a5914c4b42ed: Status 404 returned error can't find the container with id b8d1663a85e8e4d5da8c6755381d3b5a52ec1efbbf153b5783d0a5914c4b42ed
Jul 10 15:12:27.376202 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:27.376162    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x" event={"ID":"4a73c00a-d7e1-41d3-94c7-273e337b892c","Type":"ContainerStarted","Data":"c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22"}
Jul 10 15:12:27.376608 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:27.376211    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x" event={"ID":"4a73c00a-d7e1-41d3-94c7-273e337b892c","Type":"ContainerStarted","Data":"b8d1663a85e8e4d5da8c6755381d3b5a52ec1efbbf153b5783d0a5914c4b42ed"}
Jul 10 15:12:27.396417 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:27.396372    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x" podStartSLOduration=2.396357065 podStartE2EDuration="2.396357065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:12:27.395365337 +0000 UTC m=+2245.101686817" watchObservedRunningTime="2026-07-10 15:12:27.396357065 +0000 UTC m=+2245.102678586"
Jul 10 15:12:31.530332 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.530247    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6" podUID="04e09cb4-15e4-44e7-a5c3-205ec665ca95" containerName="main" containerID="cri-o://c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2" gracePeriod=45
Jul 10 15:12:31.780922 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.780868    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:12:31.817737 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.817708    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2jcw\" (UniqueName: \"kubernetes.io/projected/04e09cb4-15e4-44e7-a5c3-205ec665ca95-kube-api-access-c2jcw\") pod \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") "
Jul 10 15:12:31.817893 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.817767    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tls-certs\") pod \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") "
Jul 10 15:12:31.817893 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.817795    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-home\") pod \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") "
Jul 10 15:12:31.817893 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.817879    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tmp-dir\") pod \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") "
Jul 10 15:12:31.818055 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.817913    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-dshm\") pod \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") "
Jul 10 15:12:31.818055 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.817945    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-model-cache\") pod \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\" (UID: \"04e09cb4-15e4-44e7-a5c3-205ec665ca95\") "
Jul 10 15:12:31.818201 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.818164    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04e09cb4-15e4-44e7-a5c3-205ec665ca95" (UID: "04e09cb4-15e4-44e7-a5c3-205ec665ca95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:12:31.818412 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.818391    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tmp-dir\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:12:31.818503 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.818416    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-model-cache" (OuterVolumeSpecName: "model-cache") pod "04e09cb4-15e4-44e7-a5c3-205ec665ca95" (UID: "04e09cb4-15e4-44e7-a5c3-205ec665ca95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:12:31.818669 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.818644    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-home" (OuterVolumeSpecName: "home") pod "04e09cb4-15e4-44e7-a5c3-205ec665ca95" (UID: "04e09cb4-15e4-44e7-a5c3-205ec665ca95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:12:31.820148 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.820098    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e09cb4-15e4-44e7-a5c3-205ec665ca95-kube-api-access-c2jcw" (OuterVolumeSpecName: "kube-api-access-c2jcw") pod "04e09cb4-15e4-44e7-a5c3-205ec665ca95" (UID: "04e09cb4-15e4-44e7-a5c3-205ec665ca95"). InnerVolumeSpecName "kube-api-access-c2jcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:12:31.820376 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.820357    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-dshm" (OuterVolumeSpecName: "dshm") pod "04e09cb4-15e4-44e7-a5c3-205ec665ca95" (UID: "04e09cb4-15e4-44e7-a5c3-205ec665ca95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:12:31.820461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.820360    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04e09cb4-15e4-44e7-a5c3-205ec665ca95" (UID: "04e09cb4-15e4-44e7-a5c3-205ec665ca95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:12:31.919258 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.919230    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-dshm\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:12:31.919258 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.919254    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-model-cache\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:12:31.919258 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.919264    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2jcw\" (UniqueName: \"kubernetes.io/projected/04e09cb4-15e4-44e7-a5c3-205ec665ca95-kube-api-access-c2jcw\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:12:31.919491 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.919273    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e09cb4-15e4-44e7-a5c3-205ec665ca95-tls-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:12:31.919491 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:31.919281    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04e09cb4-15e4-44e7-a5c3-205ec665ca95-home\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:12:32.403923 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.403889    2583 generic.go:356] "Generic (PLEG): container finished" podID="04e09cb4-15e4-44e7-a5c3-205ec665ca95" containerID="c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2" exitCode=0
Jul 10 15:12:32.404085 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.403930    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"
Jul 10 15:12:32.404085 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.403972    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6" event={"ID":"04e09cb4-15e4-44e7-a5c3-205ec665ca95","Type":"ContainerDied","Data":"c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2"}
Jul 10 15:12:32.404085 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.404021    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6" event={"ID":"04e09cb4-15e4-44e7-a5c3-205ec665ca95","Type":"ContainerDied","Data":"6cbd5bf6994b9a2131f498cce3f3ad3bae3d2853fe97792177be835070f45484"}
Jul 10 15:12:32.404085 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.404042    2583 scope.go:117] "RemoveContainer" containerID="c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2"
Jul 10 15:12:32.416460 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.416439    2583 scope.go:117] "RemoveContainer" containerID="c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2"
Jul 10 15:12:32.416842 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:12:32.416817    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2\": container with ID starting with c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2 not found: ID does not exist" containerID="c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2"
Jul 10 15:12:32.416934 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.416849    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2"} err="failed to get container status \"c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2\": rpc error: code = NotFound desc = could not find container \"c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2\": container with ID starting with c860149f4417a1e807496a8056015feb2ddbaaae56b498751b8d9f7142d0b9f2 not found: ID does not exist"
Jul 10 15:12:32.428352 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.428330    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"]
Jul 10 15:12:32.433805 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.433778    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/auth-test-model-e538f1-kserve-789f7cd47f-m9pj6"]
Jul 10 15:12:32.846859 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:32.846822    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04e09cb4-15e4-44e7-a5c3-205ec665ca95" path="/var/lib/kubelet/pods/04e09cb4-15e4-44e7-a5c3-205ec665ca95/volumes"
Jul 10 15:12:36.231051 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:36.231019    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:36.231051 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:36.231060    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:36.243484 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:36.243456    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:36.438561 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:36.438511    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:12:39.391639 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:39.391600    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:12:48.140488 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:48.140449    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:12:48.244397 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:48.244358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:12:59.952749 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:12:59.952716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:13:04.371144 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:13:04.371107    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:13:29.107114 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:13:29.107077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:13:45.350611 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:13:45.350580    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"]
Jul 10 15:13:45.380876 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:13:45.380843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:13:46.239589 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:13:46.239551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:13:50.635748 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:13:50.635714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:14:00.370983 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.370899    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x" podUID="4a73c00a-d7e1-41d3-94c7-273e337b892c" containerName="main" containerID="cri-o://c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22" gracePeriod=45
Jul 10 15:14:00.640035 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.640007    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:14:00.739766 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.739731    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-dshm\") pod \"4a73c00a-d7e1-41d3-94c7-273e337b892c\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") "
Jul 10 15:14:00.739941 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.739786    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-home\") pod \"4a73c00a-d7e1-41d3-94c7-273e337b892c\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") "
Jul 10 15:14:00.739941 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.739823    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-tmp-dir\") pod \"4a73c00a-d7e1-41d3-94c7-273e337b892c\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") "
Jul 10 15:14:00.739941 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.739860    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lt5d\" (UniqueName: \"kubernetes.io/projected/4a73c00a-d7e1-41d3-94c7-273e337b892c-kube-api-access-7lt5d\") pod \"4a73c00a-d7e1-41d3-94c7-273e337b892c\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") "
Jul 10 15:14:00.739941 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.739889    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-model-cache\") pod \"4a73c00a-d7e1-41d3-94c7-273e337b892c\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") "
Jul 10 15:14:00.739941 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.739932    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a73c00a-d7e1-41d3-94c7-273e337b892c-tls-certs\") pod \"4a73c00a-d7e1-41d3-94c7-273e337b892c\" (UID: \"4a73c00a-d7e1-41d3-94c7-273e337b892c\") "
Jul 10 15:14:00.740149 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.740086    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-home" (OuterVolumeSpecName: "home") pod "4a73c00a-d7e1-41d3-94c7-273e337b892c" (UID: "4a73c00a-d7e1-41d3-94c7-273e337b892c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:14:00.740149 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.740123    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a73c00a-d7e1-41d3-94c7-273e337b892c" (UID: "4a73c00a-d7e1-41d3-94c7-273e337b892c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:14:00.740280 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.740262    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-home\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:14:00.740334 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.740286    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-tmp-dir\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:14:00.740334 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.740264    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a73c00a-d7e1-41d3-94c7-273e337b892c" (UID: "4a73c00a-d7e1-41d3-94c7-273e337b892c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:14:00.742134 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.742113    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-dshm" (OuterVolumeSpecName: "dshm") pod "4a73c00a-d7e1-41d3-94c7-273e337b892c" (UID: "4a73c00a-d7e1-41d3-94c7-273e337b892c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:14:00.742134 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.742111    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a73c00a-d7e1-41d3-94c7-273e337b892c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a73c00a-d7e1-41d3-94c7-273e337b892c" (UID: "4a73c00a-d7e1-41d3-94c7-273e337b892c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:14:00.742282 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.742175    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a73c00a-d7e1-41d3-94c7-273e337b892c-kube-api-access-7lt5d" (OuterVolumeSpecName: "kube-api-access-7lt5d") pod "4a73c00a-d7e1-41d3-94c7-273e337b892c" (UID: "4a73c00a-d7e1-41d3-94c7-273e337b892c"). InnerVolumeSpecName "kube-api-access-7lt5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:14:00.841170 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.841139    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-model-cache\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:14:00.841170 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.841166    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a73c00a-d7e1-41d3-94c7-273e337b892c-tls-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:14:00.841170 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.841174    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a73c00a-d7e1-41d3-94c7-273e337b892c-dshm\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:14:00.841398 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.841183    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lt5d\" (UniqueName: \"kubernetes.io/projected/4a73c00a-d7e1-41d3-94c7-273e337b892c-kube-api-access-7lt5d\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:14:00.866358 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.866328    2583 generic.go:356] "Generic (PLEG): container finished" podID="4a73c00a-d7e1-41d3-94c7-273e337b892c" containerID="c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22" exitCode=0
Jul 10 15:14:00.866536 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.866365    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"
Jul 10 15:14:00.866536 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.866386    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x" event={"ID":"4a73c00a-d7e1-41d3-94c7-273e337b892c","Type":"ContainerDied","Data":"c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22"}
Jul 10 15:14:00.866536 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.866416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x" event={"ID":"4a73c00a-d7e1-41d3-94c7-273e337b892c","Type":"ContainerDied","Data":"b8d1663a85e8e4d5da8c6755381d3b5a52ec1efbbf153b5783d0a5914c4b42ed"}
Jul 10 15:14:00.866536 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.866431    2583 scope.go:117] "RemoveContainer" containerID="c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22"
Jul 10 15:14:00.875674 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.875650    2583 scope.go:117] "RemoveContainer" containerID="c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22"
Jul 10 15:14:00.875950 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:14:00.875931    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22\": container with ID starting with c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22 not found: ID does not exist" containerID="c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22"
Jul 10 15:14:00.876008 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.875960    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22"} err="failed to get container status \"c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22\": rpc error: code = NotFound desc = could not find container \"c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22\": container with ID starting with c4284d151c11c53841020d6624596b17829dd9758bce97bbdc4666487a508a22 not found: ID does not exist"
Jul 10 15:14:00.903898 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.903866    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"]
Jul 10 15:14:00.910591 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:00.910563    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/sub-test-model-7b13c8-kserve-7dc798d99f-5sw9x"]
Jul 10 15:14:02.846593 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:02.846560    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a73c00a-d7e1-41d3-94c7-273e337b892c" path="/var/lib/kubelet/pods/4a73c00a-d7e1-41d3-94c7-273e337b892c/volumes"
Jul 10 15:14:05.096684 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.096643    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"]
Jul 10 15:14:05.097263 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.097240    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e09cb4-15e4-44e7-a5c3-205ec665ca95" containerName="main"
Jul 10 15:14:05.097349 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.097265    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e09cb4-15e4-44e7-a5c3-205ec665ca95" containerName="main"
Jul 10 15:14:05.097349 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.097297    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a73c00a-d7e1-41d3-94c7-273e337b892c" containerName="main"
Jul 10 15:14:05.097349 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.097306    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a73c00a-d7e1-41d3-94c7-273e337b892c" containerName="main"
Jul 10 15:14:05.097506 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.097418    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="04e09cb4-15e4-44e7-a5c3-205ec665ca95" containerName="main"
Jul 10 15:14:05.097506 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.097435    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a73c00a-d7e1-41d3-94c7-273e337b892c" containerName="main"
Jul 10 15:14:05.102004 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.101982    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.105066 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.105042    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"default-dockercfg-9mkjk\""
Jul 10 15:14:05.106132 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.106112    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"kube-root-ca.crt\""
Jul 10 15:14:05.106132 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.106122    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"rate-test-model-7b13c8-kserve-self-signed-certs\""
Jul 10 15:14:05.106277 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.106162    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"openshift-service-ca.crt\""
Jul 10 15:14:05.119927 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.119901    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"]
Jul 10 15:14:05.131435 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.131411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:05.179448 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.179414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tls-certs\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.179650 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.179461    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv6wd\" (UniqueName: \"kubernetes.io/projected/3c2a9d56-b1d2-4325-a039-b226e02e2dea-kube-api-access-bv6wd\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.179650 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.179546    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-model-cache\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.179650 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.179599    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tmp-dir\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.179650 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.179635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-dshm\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.179795 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.179659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-home\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281075 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-home\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281242 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281090    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tls-certs\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281242 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281126    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv6wd\" (UniqueName: \"kubernetes.io/projected/3c2a9d56-b1d2-4325-a039-b226e02e2dea-kube-api-access-bv6wd\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281242 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-model-cache\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281242 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tmp-dir\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281242 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281215    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-dshm\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281584 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281493    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-home\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281584 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281506    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-model-cache\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.281680 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.281628    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tmp-dir\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.283453 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.283431    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-dshm\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.283673 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.283652    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tls-certs\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.292007 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.291984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv6wd\" (UniqueName: \"kubernetes.io/projected/3c2a9d56-b1d2-4325-a039-b226e02e2dea-kube-api-access-bv6wd\") pod \"rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") " pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.412121 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.412092    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:05.570938 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.570777    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"]
Jul 10 15:14:05.573778 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:14:05.573747    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c2a9d56_b1d2_4325_a039_b226e02e2dea.slice/crio-272b39636cf1a0195fb19dd3adc2221a8c78a8b2ecad801195f97afb803268bc WatchSource:0}: Error finding container 272b39636cf1a0195fb19dd3adc2221a8c78a8b2ecad801195f97afb803268bc: Status 404 returned error can't find the container with id 272b39636cf1a0195fb19dd3adc2221a8c78a8b2ecad801195f97afb803268bc
Jul 10 15:14:05.904734 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.904633    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z" event={"ID":"3c2a9d56-b1d2-4325-a039-b226e02e2dea","Type":"ContainerStarted","Data":"07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4"}
Jul 10 15:14:05.904734 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.904690    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z" event={"ID":"3c2a9d56-b1d2-4325-a039-b226e02e2dea","Type":"ContainerStarted","Data":"272b39636cf1a0195fb19dd3adc2221a8c78a8b2ecad801195f97afb803268bc"}
Jul 10 15:14:05.931447 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:05.931396    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z" podStartSLOduration=0.931381788 podStartE2EDuration="931.381788ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:14:05.930369275 +0000 UTC m=+2343.636690744" watchObservedRunningTime="2026-07-10 15:14:05.931381788 +0000 UTC m=+2343.637703259"
Jul 10 15:14:09.077185 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:09.077148    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:14:09.242355 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:09.242316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:09.347741 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:09.347607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:10.798615 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:10.798573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:10.888029 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:10.887994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:15.412419 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:15.412376    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:15.412897 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:15.412435    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:15.425231 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:15.425196    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:15.969205 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:15.969175    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:14:19.451698 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:19.451665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:21.832272 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:21.832234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:14:26.867942 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:26.867911    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:14:41.261257 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:41.261224    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:14:58.068166 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:14:58.068133    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:15:00.144749 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.144718    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds"]
Jul 10 15:15:00.148416 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.148398    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"]
Jul 10 15:15:00.148545 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.148507    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds"
Jul 10 15:15:00.150950 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.150929    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gj49t\""
Jul 10 15:15:00.151866 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.151851    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"
Jul 10 15:15:00.158115 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.158089    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"]
Jul 10 15:15:00.160635 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.160613    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds"]
Jul 10 15:15:00.162545 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.162527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds"
Jul 10 15:15:00.173052 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.173033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"
Jul 10 15:15:00.316657 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.316620    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds"]
Jul 10 15:15:00.319565 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:15:00.319535    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd761155_2346_46c5_9655_f911cafd9b24.slice/crio-63a8e4c459ca4dd67f168e80438e1f8d6cf084a8c846e4c0665504945841dba8 WatchSource:0}: Error finding container 63a8e4c459ca4dd67f168e80438e1f8d6cf084a8c846e4c0665504945841dba8: Status 404 returned error can't find the container with id 63a8e4c459ca4dd67f168e80438e1f8d6cf084a8c846e4c0665504945841dba8
Jul 10 15:15:00.344598 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.344571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"]
Jul 10 15:15:00.345328 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:15:00.345305    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0cad14_49bc_41ac_b3fb_f326f3973105.slice/crio-4819f01d38f4b6fc2e6f7de711fc49223dd20ea97cbfb281fc692164c5e84287 WatchSource:0}: Error finding container 4819f01d38f4b6fc2e6f7de711fc49223dd20ea97cbfb281fc692164c5e84287: Status 404 returned error can't find the container with id 4819f01d38f4b6fc2e6f7de711fc49223dd20ea97cbfb281fc692164c5e84287
Jul 10 15:15:00.680960 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.680860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:00.781608 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:00.781511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:01.198747 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:01.198703    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt" event={"ID":"2a0cad14-49bc-41ac-b3fb-f326f3973105","Type":"ContainerStarted","Data":"f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd"}
Jul 10 15:15:01.199185 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:01.198802    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt" event={"ID":"2a0cad14-49bc-41ac-b3fb-f326f3973105","Type":"ContainerStarted","Data":"4819f01d38f4b6fc2e6f7de711fc49223dd20ea97cbfb281fc692164c5e84287"}
Jul 10 15:15:01.200310 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:01.200273    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds" event={"ID":"dd761155-2346-46c5-9655-f911cafd9b24","Type":"ContainerStarted","Data":"3b512aecd11eb818f40a3ba50b5794c5efff44398a53b5bb7aa00faba90407fa"}
Jul 10 15:15:01.200434 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:01.200314    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds" event={"ID":"dd761155-2346-46c5-9655-f911cafd9b24","Type":"ContainerStarted","Data":"63a8e4c459ca4dd67f168e80438e1f8d6cf084a8c846e4c0665504945841dba8"}
Jul 10 15:15:01.218406 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:01.218349    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt" podStartSLOduration=1.218331333 podStartE2EDuration="1.218331333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:15:01.216159127 +0000 UTC m=+2398.922480600" watchObservedRunningTime="2026-07-10 15:15:01.218331333 +0000 UTC m=+2398.924652800"
Jul 10 15:15:01.233788 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:01.233743    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds" podStartSLOduration=1.23372571 podStartE2EDuration="1.23372571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:15:01.232125056 +0000 UTC m=+2398.938446525" watchObservedRunningTime="2026-07-10 15:15:01.23372571 +0000 UTC m=+2398.940047180"
Jul 10 15:15:03.198132 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:03.198099    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:15:03.203729 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:03.203707    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8565_43fa7fd7-d317-44b9-91b8-23fb1f464ad2/ovn-acl-logging/0.log"
Jul 10 15:15:04.447105 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:04.447063    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"]
Jul 10 15:15:04.486117 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:04.486084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:05.417341 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:05.417306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:07.104377 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:07.104339    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:07.205119 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:07.205079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:11.466631 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:11.466595    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:15:19.467396 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.467359    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z" podUID="3c2a9d56-b1d2-4325-a039-b226e02e2dea" containerName="main" containerID="cri-o://07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4" gracePeriod=45
Jul 10 15:15:19.732717 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.732656    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:15:19.866218 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866173    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tmp-dir\") pod \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") "
Jul 10 15:15:19.866218 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866232    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-home\") pod \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") "
Jul 10 15:15:19.866476 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866297    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv6wd\" (UniqueName: \"kubernetes.io/projected/3c2a9d56-b1d2-4325-a039-b226e02e2dea-kube-api-access-bv6wd\") pod \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") "
Jul 10 15:15:19.866476 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866328    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-model-cache\") pod \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") "
Jul 10 15:15:19.866476 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866370    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-dshm\") pod \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") "
Jul 10 15:15:19.866476 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866388    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tls-certs\") pod \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\" (UID: \"3c2a9d56-b1d2-4325-a039-b226e02e2dea\") "
Jul 10 15:15:19.866734 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866509    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-home" (OuterVolumeSpecName: "home") pod "3c2a9d56-b1d2-4325-a039-b226e02e2dea" (UID: "3c2a9d56-b1d2-4325-a039-b226e02e2dea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:15:19.866734 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866545    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c2a9d56-b1d2-4325-a039-b226e02e2dea" (UID: "3c2a9d56-b1d2-4325-a039-b226e02e2dea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:15:19.866734 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866710    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c2a9d56-b1d2-4325-a039-b226e02e2dea" (UID: "3c2a9d56-b1d2-4325-a039-b226e02e2dea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:15:19.866862 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866803    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tmp-dir\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:19.866862 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.866827    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-home\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:19.868656 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.868621    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c2a9d56-b1d2-4325-a039-b226e02e2dea" (UID: "3c2a9d56-b1d2-4325-a039-b226e02e2dea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:15:19.868790 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.868662    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c2a9d56-b1d2-4325-a039-b226e02e2dea-kube-api-access-bv6wd" (OuterVolumeSpecName: "kube-api-access-bv6wd") pod "3c2a9d56-b1d2-4325-a039-b226e02e2dea" (UID: "3c2a9d56-b1d2-4325-a039-b226e02e2dea"). InnerVolumeSpecName "kube-api-access-bv6wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:15:19.868790 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.868686    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-dshm" (OuterVolumeSpecName: "dshm") pod "3c2a9d56-b1d2-4325-a039-b226e02e2dea" (UID: "3c2a9d56-b1d2-4325-a039-b226e02e2dea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:15:19.967725 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.967678    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv6wd\" (UniqueName: \"kubernetes.io/projected/3c2a9d56-b1d2-4325-a039-b226e02e2dea-kube-api-access-bv6wd\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:19.967725 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.967720    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-model-cache\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:19.967725 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.967731    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2a9d56-b1d2-4325-a039-b226e02e2dea-dshm\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:19.967725 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:19.967739    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2a9d56-b1d2-4325-a039-b226e02e2dea-tls-certs\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:20.304294 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.304254    2583 generic.go:356] "Generic (PLEG): container finished" podID="3c2a9d56-b1d2-4325-a039-b226e02e2dea" containerID="07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4" exitCode=0
Jul 10 15:15:20.304471 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.304316    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"
Jul 10 15:15:20.304471 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.304332    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z" event={"ID":"3c2a9d56-b1d2-4325-a039-b226e02e2dea","Type":"ContainerDied","Data":"07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4"}
Jul 10 15:15:20.304471 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.304378    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z" event={"ID":"3c2a9d56-b1d2-4325-a039-b226e02e2dea","Type":"ContainerDied","Data":"272b39636cf1a0195fb19dd3adc2221a8c78a8b2ecad801195f97afb803268bc"}
Jul 10 15:15:20.304471 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.304395    2583 scope.go:117] "RemoveContainer" containerID="07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4"
Jul 10 15:15:20.314035 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.314016    2583 scope.go:117] "RemoveContainer" containerID="07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4"
Jul 10 15:15:20.314307 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:15:20.314287    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4\": container with ID starting with 07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4 not found: ID does not exist" containerID="07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4"
Jul 10 15:15:20.314348 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.314316    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4"} err="failed to get container status \"07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4\": rpc error: code = NotFound desc = could not find container \"07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4\": container with ID starting with 07a3526f91c71ddc63f7f6184cd8ef21f8fdbda2bede7a9030e72ae105555fe4 not found: ID does not exist"
Jul 10 15:15:20.353387 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.353356    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"]
Jul 10 15:15:20.361362 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.361340    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/rate-test-model-7b13c8-kserve-79c956dc5f-jzv7z"]
Jul 10 15:15:20.847382 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:20.847349    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c2a9d56-b1d2-4325-a039-b226e02e2dea" path="/var/lib/kubelet/pods/3c2a9d56-b1d2-4325-a039-b226e02e2dea/volumes"
Jul 10 15:15:21.510800 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.510767    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"]
Jul 10 15:15:21.511235 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.511223    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c2a9d56-b1d2-4325-a039-b226e02e2dea" containerName="main"
Jul 10 15:15:21.511278 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.511236    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c2a9d56-b1d2-4325-a039-b226e02e2dea" containerName="main"
Jul 10 15:15:21.511313 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.511308    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c2a9d56-b1d2-4325-a039-b226e02e2dea" containerName="main"
Jul 10 15:15:21.515824 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.515807    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:21.518440 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.518421    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wprz9\""
Jul 10 15:15:21.523808 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.523783    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"]
Jul 10 15:15:21.684881 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.684843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtkw\" (UniqueName: \"kubernetes.io/projected/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e-kube-api-access-wbtkw\") pod \"test-curl-7741-139770259762032\" (UID: \"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e\") " pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:21.785438 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.785345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtkw\" (UniqueName: \"kubernetes.io/projected/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e-kube-api-access-wbtkw\") pod \"test-curl-7741-139770259762032\" (UID: \"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e\") " pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:21.795385 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.795360    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtkw\" (UniqueName: \"kubernetes.io/projected/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e-kube-api-access-wbtkw\") pod \"test-curl-7741-139770259762032\" (UID: \"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e\") " pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:21.826026 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.825999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:21.964329 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.964296    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"]
Jul 10 15:15:21.967944 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:15:21.967913    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86dd1db4_98ff_42b7_a2b9_7752b4b70f0e.slice/crio-dbd028ca477916114c8fe0c5198e02e63fc0d4944009e0510db9e512407f05b4 WatchSource:0}: Error finding container dbd028ca477916114c8fe0c5198e02e63fc0d4944009e0510db9e512407f05b4: Status 404 returned error can't find the container with id dbd028ca477916114c8fe0c5198e02e63fc0d4944009e0510db9e512407f05b4
Jul 10 15:15:21.969273 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:21.969253    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 15:15:22.317786 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:22.317750    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259762032" event={"ID":"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e","Type":"ContainerStarted","Data":"dbd028ca477916114c8fe0c5198e02e63fc0d4944009e0510db9e512407f05b4"}
Jul 10 15:15:23.324346 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:23.324313    2583 generic.go:356] "Generic (PLEG): container finished" podID="86dd1db4-98ff-42b7-a2b9-7752b4b70f0e" containerID="c62a6642da4e31d82567ab86e017497eb7600e8fb79c50b7763ed3b6141ca470" exitCode=0
Jul 10 15:15:23.324836 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:23.324404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259762032" event={"ID":"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e","Type":"ContainerDied","Data":"c62a6642da4e31d82567ab86e017497eb7600e8fb79c50b7763ed3b6141ca470"}
Jul 10 15:15:24.475198 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.475172    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:24.497072 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.497041    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7741-139770259762032_86dd1db4-98ff-42b7-a2b9-7752b4b70f0e/test-curl-7741-139770259762032/0.log"
Jul 10 15:15:24.523656 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.523628    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"]
Jul 10 15:15:24.530435 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.530404    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"]
Jul 10 15:15:24.611191 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.611121    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbtkw\" (UniqueName: \"kubernetes.io/projected/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e-kube-api-access-wbtkw\") pod \"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e\" (UID: \"86dd1db4-98ff-42b7-a2b9-7752b4b70f0e\") "
Jul 10 15:15:24.613155 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.613125    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e-kube-api-access-wbtkw" (OuterVolumeSpecName: "kube-api-access-wbtkw") pod "86dd1db4-98ff-42b7-a2b9-7752b4b70f0e" (UID: "86dd1db4-98ff-42b7-a2b9-7752b4b70f0e"). InnerVolumeSpecName "kube-api-access-wbtkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:15:24.712293 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.712257    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbtkw\" (UniqueName: \"kubernetes.io/projected/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e-kube-api-access-wbtkw\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:24.754691 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.754657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:24.846733 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:24.846705    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86dd1db4-98ff-42b7-a2b9-7752b4b70f0e" path="/var/lib/kubelet/pods/86dd1db4-98ff-42b7-a2b9-7752b4b70f0e/volumes"
Jul 10 15:15:25.156764 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.156639    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:25.257197 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.257165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:25.336964 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.336940    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032"
Jul 10 15:15:25.337127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.336975    2583 scope.go:117] "RemoveContainer" containerID="c62a6642da4e31d82567ab86e017497eb7600e8fb79c50b7763ed3b6141ca470"
Jul 10 15:15:25.361656 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.361627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:25.592103 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.592069    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"]
Jul 10 15:15:25.592563 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.592318    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" podUID="ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" containerName="maas-api" containerID="cri-o://8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba" gracePeriod=30
Jul 10 15:15:25.688548 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.688495    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:25.851837 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:25.851780    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:15:26.025380 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.025354    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h86fz\" (UniqueName: \"kubernetes.io/projected/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-kube-api-access-h86fz\") pod \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") "
Jul 10 15:15:26.025559 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.025444    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls\") pod \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") "
Jul 10 15:15:26.025625 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.025550    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-openshift-service-ca\") pod \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\" (UID: \"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3\") "
Jul 10 15:15:26.025944 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.025900    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" (UID: "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:15:26.027967 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.027949    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-kube-api-access-h86fz" (OuterVolumeSpecName: "kube-api-access-h86fz") pod "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" (UID: "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3"). InnerVolumeSpecName "kube-api-access-h86fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:15:26.027967 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.027953    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" (UID: "ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:15:26.126864 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.126779    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h86fz\" (UniqueName: \"kubernetes.io/projected/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-kube-api-access-h86fz\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:26.126864 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.126808    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:26.126864 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.126820    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:26.344780 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.344745    2583 generic.go:356] "Generic (PLEG): container finished" podID="ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" containerID="8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba" exitCode=0
Jul 10 15:15:26.344940 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.344787    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"
Jul 10 15:15:26.344940 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.344831    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" event={"ID":"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3","Type":"ContainerDied","Data":"8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba"}
Jul 10 15:15:26.344940 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.344884    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd" event={"ID":"ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3","Type":"ContainerDied","Data":"2a760c57002f3f05d61cffc6bdf46e765391025e20f11021e5bb60a10b7b7704"}
Jul 10 15:15:26.344940 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.344910    2583 scope.go:117] "RemoveContainer" containerID="8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba"
Jul 10 15:15:26.357863 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.357847    2583 scope.go:117] "RemoveContainer" containerID="8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba"
Jul 10 15:15:26.358135 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:15:26.358117    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba\": container with ID starting with 8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba not found: ID does not exist" containerID="8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba"
Jul 10 15:15:26.358198 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.358145    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba"} err="failed to get container status \"8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba\": rpc error: code = NotFound desc = could not find container \"8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba\": container with ID starting with 8e1dd64e7516ca0c5634a59137e63cb34e0a9e4bcaf0e104f1b25caf9ca57cba not found: ID does not exist"
Jul 10 15:15:26.368916 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.368893    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"]
Jul 10 15:15:26.373178 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.373157    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b13c8-699865d8df-ppwgd"]
Jul 10 15:15:26.846458 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:26.846424    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" path="/var/lib/kubelet/pods/ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3/volumes"
Jul 10 15:15:29.150779 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:29.150746    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:15:31.470044 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:31.469998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:31.534020 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:31.533979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:31.791586 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:31.791481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:31.969264 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:31.969232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:32.148529 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.148414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:32.519386 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.519354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:32.619695 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.619661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:32.720050 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.720012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:32.873350 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.873267    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"]
Jul 10 15:15:32.873575 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.873501    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9" podUID="b3fdcd28-f6d0-4ea4-98fc-d139714ea282" containerName="maas-api" containerID="cri-o://5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b" gracePeriod=30
Jul 10 15:15:32.912930 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.912901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:32.921258 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.921222    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"]
Jul 10 15:15:32.921417 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:32.921399    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt" podUID="2a0cad14-49bc-41ac-b3fb-f326f3973105" containerName="cleanup" containerID="cri-o://f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd" gracePeriod=30
Jul 10 15:15:33.332586 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.332559    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:15:33.382618 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.382580    2583 generic.go:356] "Generic (PLEG): container finished" podID="b3fdcd28-f6d0-4ea4-98fc-d139714ea282" containerID="5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b" exitCode=0
Jul 10 15:15:33.382618 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.382624    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"
Jul 10 15:15:33.382871 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.382654    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9" event={"ID":"b3fdcd28-f6d0-4ea4-98fc-d139714ea282","Type":"ContainerDied","Data":"5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b"}
Jul 10 15:15:33.382871 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.382723    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9" event={"ID":"b3fdcd28-f6d0-4ea4-98fc-d139714ea282","Type":"ContainerDied","Data":"bc7030183f391cd083065357fb4e1ba836f9266f81f1a89e32077519f9589e46"}
Jul 10 15:15:33.382871 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.382757    2583 scope.go:117] "RemoveContainer" containerID="5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b"
Jul 10 15:15:33.392407 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.392385    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnhbl\" (UniqueName: \"kubernetes.io/projected/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-kube-api-access-vnhbl\") pod \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") "
Jul 10 15:15:33.392554 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.392428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-maas-api-tls\") pod \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") "
Jul 10 15:15:33.392554 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.392546    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-openshift-service-ca\") pod \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\" (UID: \"b3fdcd28-f6d0-4ea4-98fc-d139714ea282\") "
Jul 10 15:15:33.392765 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.392569    2583 scope.go:117] "RemoveContainer" containerID="5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b"
Jul 10 15:15:33.392881 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.392852    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3fdcd28-f6d0-4ea4-98fc-d139714ea282" (UID: "b3fdcd28-f6d0-4ea4-98fc-d139714ea282"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:15:33.393005 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:15:33.392911    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b\": container with ID starting with 5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b not found: ID does not exist" containerID="5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b"
Jul 10 15:15:33.393005 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.392938    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b"} err="failed to get container status \"5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b\": rpc error: code = NotFound desc = could not find container \"5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b\": container with ID starting with 5f6c135566108f5ffae1efab9d3e040d844e05398c4861bb86741005f051fa3b not found: ID does not exist"
Jul 10 15:15:33.394594 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.394569    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3fdcd28-f6d0-4ea4-98fc-d139714ea282" (UID: "b3fdcd28-f6d0-4ea4-98fc-d139714ea282"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:15:33.394731 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.394626    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-kube-api-access-vnhbl" (OuterVolumeSpecName: "kube-api-access-vnhbl") pod "b3fdcd28-f6d0-4ea4-98fc-d139714ea282" (UID: "b3fdcd28-f6d0-4ea4-98fc-d139714ea282"). InnerVolumeSpecName "kube-api-access-vnhbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:15:33.494138 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.494109    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-openshift-service-ca\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:33.494138 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.494135    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnhbl\" (UniqueName: \"kubernetes.io/projected/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-kube-api-access-vnhbl\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:33.494138 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.494145    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fdcd28-f6d0-4ea4-98fc-d139714ea282-maas-api-tls\") on node \"ip-10-0-136-210.ec2.internal\" DevicePath \"\""
Jul 10 15:15:33.706447 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.706414    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"]
Jul 10 15:15:33.710131 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:33.710105    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e538f1-54576ddf74-hdts9"]
Jul 10 15:15:34.846381 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:34.846347    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3fdcd28-f6d0-4ea4-98fc-d139714ea282" path="/var/lib/kubelet/pods/b3fdcd28-f6d0-4ea4-98fc-d139714ea282/volumes"
Jul 10 15:15:38.764952 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:38.764918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:38.833461 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:38.833424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:39.038651 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:39.038566    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:39.241190 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:39.241154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:40.859367 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:40.859327    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2987p"]
Jul 10 15:15:44.385096 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.385064    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mq8jn_be021526-cbe6-4f16-a1ac-50d9c38b12ac/manager/0.log"
Jul 10 15:15:44.505955 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.505924    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dl6t6_f23f7574-ffe1-474e-b36c-617f367240b0/manager/0.log"
Jul 10 15:15:44.629754 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.629724    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b89cd5547-8cpt7_cd521ab1-8096-47e5-91f9-782408cf0dc2/manager/0.log"
Jul 10 15:15:44.743837 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.743808    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rqg5c_b85bd97c-19c6-45ea-9fa7-3505181a3293/server/0.log"
Jul 10 15:15:44.857933 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.857898    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zbx64_51a6fdb3-a4da-4ac3-b8b3-5f234ade7048/manager/0.log"
Jul 10 15:15:44.975418 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.975390    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-8xmv4_e9febbf1-8597-481a-84f2-c344b636cd41/copy-manifests/0.log"
Jul 10 15:15:44.983315 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:44.983285    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-8xmv4_e9febbf1-8597-481a-84f2-c344b636cd41/manager/0.log"
Jul 10 15:15:45.577260 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:45.577230    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5948dcb657-7q2rk_efa59778-99ff-4210-a681-91dd3b970a9e/maas-api/0.log"
Jul 10 15:15:45.683704 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:45.683676    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29728275-s8pds_dd761155-2346-46c5-9655-f911cafd9b24/cleanup/0.log"
Jul 10 15:15:45.903205 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:45.903122    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt_2a0cad14-49bc-41ac-b3fb-f326f3973105/cleanup/0.log"
Jul 10 15:15:46.009047 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:46.009012    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-tbp4z_fd7f7bf4-3555-445e-91ec-15413bbd5cdb/postgres/0.log"
Jul 10 15:15:47.392022 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:47.391992    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7cd7q_7a696a6b-9f3b-4d25-b663-83e67a2636ad/manager/0.log"
Jul 10 15:15:47.504152 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:47.504118    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rfwjs_fba938ec-4ea9-4596-858d-50490b870d0c/manager/0.log"
Jul 10 15:15:47.735298 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:47.735267    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rbh9q_e2a3590f-5df2-484b-834d-e5e1636ec674/registry-server/0.log"
Jul 10 15:15:47.975704 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:47.975674    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2987p_a908618e-c152-4b10-9f50-3037bfec9cf5/limitador/0.log"
Jul 10 15:15:48.092353 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:48.092265    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-blxxm_26e00d4a-b2af-4e38-8db8-72763b0e066f/manager/0.log"
Jul 10 15:15:48.777995 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:48.777965    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-p2dhc_0186c81d-ae51-4005-81fe-ca87633d1494/kube-auth-proxy/0.log"
Jul 10 15:15:48.896013 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:48.895986    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/1.log"
Jul 10 15:15:49.011337 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:49.011312    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gwjx9_b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f/payload-pre-processing/0.log"
Jul 10 15:15:49.572929 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:49.572896    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mlkxj_e50010f1-6ae8-4cc6-aeaa-6e010c30923b/main/0.log"
Jul 10 15:15:49.796150 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:49.796117    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-84xck_bbebd686-8880-41c3-a764-f1ef1beea8e9/main/0.log"
Jul 10 15:15:54.156712 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.156675    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c6l7l/must-gather-m6xqb"]
Jul 10 15:15:54.157354 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157333    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86dd1db4-98ff-42b7-a2b9-7752b4b70f0e" containerName="test-curl-7741-139770259762032"
Jul 10 15:15:54.157442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157357    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="86dd1db4-98ff-42b7-a2b9-7752b4b70f0e" containerName="test-curl-7741-139770259762032"
Jul 10 15:15:54.157442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157375    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3fdcd28-f6d0-4ea4-98fc-d139714ea282" containerName="maas-api"
Jul 10 15:15:54.157442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157383    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3fdcd28-f6d0-4ea4-98fc-d139714ea282" containerName="maas-api"
Jul 10 15:15:54.157442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157405    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" containerName="maas-api"
Jul 10 15:15:54.157442 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157414    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" containerName="maas-api"
Jul 10 15:15:54.157742 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157539    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea9859c6-94a9-4e4a-bb0d-f5c6d2e171f3" containerName="maas-api"
Jul 10 15:15:54.157742 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157552    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="86dd1db4-98ff-42b7-a2b9-7752b4b70f0e" containerName="test-curl-7741-139770259762032"
Jul 10 15:15:54.157742 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.157570    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3fdcd28-f6d0-4ea4-98fc-d139714ea282" containerName="maas-api"
Jul 10 15:15:54.161174 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.161150    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.163834 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.163808    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c6l7l\"/\"default-dockercfg-qchrr\""
Jul 10 15:15:54.163963 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.163844    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6l7l\"/\"openshift-service-ca.crt\""
Jul 10 15:15:54.163963 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.163912    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6l7l\"/\"kube-root-ca.crt\""
Jul 10 15:15:54.171430 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.171407    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6l7l/must-gather-m6xqb"]
Jul 10 15:15:54.198045 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.198009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:54.280555 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.280499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa55c841-6ffb-4903-9ead-0af7a53105f1-must-gather-output\") pod \"must-gather-m6xqb\" (UID: \"aa55c841-6ffb-4903-9ead-0af7a53105f1\") " pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.280710 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.280622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5jwg\" (UniqueName: \"kubernetes.io/projected/aa55c841-6ffb-4903-9ead-0af7a53105f1-kube-api-access-l5jwg\") pod \"must-gather-m6xqb\" (UID: \"aa55c841-6ffb-4903-9ead-0af7a53105f1\") " pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.297315 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.297289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log" limit=1024
Jul 10 15:15:54.381280 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.381246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa55c841-6ffb-4903-9ead-0af7a53105f1-must-gather-output\") pod \"must-gather-m6xqb\" (UID: \"aa55c841-6ffb-4903-9ead-0af7a53105f1\") " pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.381495 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.381314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5jwg\" (UniqueName: \"kubernetes.io/projected/aa55c841-6ffb-4903-9ead-0af7a53105f1-kube-api-access-l5jwg\") pod \"must-gather-m6xqb\" (UID: \"aa55c841-6ffb-4903-9ead-0af7a53105f1\") " pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.381657 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.381637    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa55c841-6ffb-4903-9ead-0af7a53105f1-must-gather-output\") pod \"must-gather-m6xqb\" (UID: \"aa55c841-6ffb-4903-9ead-0af7a53105f1\") " pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.391265 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.391234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5jwg\" (UniqueName: \"kubernetes.io/projected/aa55c841-6ffb-4903-9ead-0af7a53105f1-kube-api-access-l5jwg\") pod \"must-gather-m6xqb\" (UID: \"aa55c841-6ffb-4903-9ead-0af7a53105f1\") " pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.471080 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.471040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6l7l/must-gather-m6xqb"
Jul 10 15:15:54.618604 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:54.618576    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6l7l/must-gather-m6xqb"]
Jul 10 15:15:54.621382 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:15:54.621338    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa55c841_6ffb_4903_9ead_0af7a53105f1.slice/crio-22d6a4fe7966731562e16d3fc99edb3b8cae4b64e84bc65ad18838e4b6854a2a WatchSource:0}: Error finding container 22d6a4fe7966731562e16d3fc99edb3b8cae4b64e84bc65ad18838e4b6854a2a: Status 404 returned error can't find the container with id 22d6a4fe7966731562e16d3fc99edb3b8cae4b64e84bc65ad18838e4b6854a2a
Jul 10 15:15:55.501919 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:55.501873    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/must-gather-m6xqb" event={"ID":"aa55c841-6ffb-4903-9ead-0af7a53105f1","Type":"ContainerStarted","Data":"c5f0ac7d2462820ed5e1610a5a39d0e931007e4dcb1b4e033c7304ea0de8912c"}
Jul 10 15:15:55.502226 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:55.501929    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/must-gather-m6xqb" event={"ID":"aa55c841-6ffb-4903-9ead-0af7a53105f1","Type":"ContainerStarted","Data":"22d6a4fe7966731562e16d3fc99edb3b8cae4b64e84bc65ad18838e4b6854a2a"}
Jul 10 15:15:56.513138 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:56.512202    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/must-gather-m6xqb" event={"ID":"aa55c841-6ffb-4903-9ead-0af7a53105f1","Type":"ContainerStarted","Data":"2e1a816d506a675edeeecac8bddbb47a6b423991c4ffb65bece9f850eb5cbe88"}
Jul 10 15:15:56.531434 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:56.531355    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c6l7l/must-gather-m6xqb" podStartSLOduration=1.75474099 podStartE2EDuration="2.531337581s" totalImagesPullingTime="776.596591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:15:56.530914501 +0000 UTC m=+2454.237235972" watchObservedRunningTime="2026-07-10 15:15:56.531337581 +0000 UTC m=+2454.237659078"
Jul 10 15:15:57.006094 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:57.006046    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-clcb4_ba9c54c3-88af-4d02-8935-2a8c8eeb5dbf/global-pull-secret-syncer/0.log"
Jul 10 15:15:57.145391 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:57.145356    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-25t26_9c33d692-b328-4356-982f-03e784e200a3/konnectivity-agent/0.log"
Jul 10 15:15:57.263701 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:15:57.263624    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-210.ec2.internal_3d95b182bc9232c53452fc18d92789eb/haproxy/0.log"
Jul 10 15:16:00.013910 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:00.013869    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds"]
Jul 10 15:16:00.014754 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:00.014215    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728275-s8pds" podUID="dd761155-2346-46c5-9655-f911cafd9b24" containerName="cleanup" containerID="cri-o://3b512aecd11eb818f40a3ba50b5794c5efff44398a53b5bb7aa00faba90407fa" gracePeriod=30
Jul 10 15:16:02.534509 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:02.534463    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7cd7q_7a696a6b-9f3b-4d25-b663-83e67a2636ad/manager/0.log"
Jul 10 15:16:02.569198 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:02.569170    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rfwjs_fba938ec-4ea9-4596-858d-50490b870d0c/manager/0.log"
Jul 10 15:16:02.643570 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:02.643540    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rbh9q_e2a3590f-5df2-484b-834d-e5e1636ec674/registry-server/0.log"
Jul 10 15:16:02.882616 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:02.882582    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2987p_a908618e-c152-4b10-9f50-3037bfec9cf5/limitador/0.log"
Jul 10 15:16:02.976127 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:02.975294    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-blxxm_26e00d4a-b2af-4e38-8db8-72763b0e066f/manager/0.log"
Jul 10 15:16:03.218317 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.218269    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"
Jul 10 15:16:03.572174 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.572088    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a0cad14-49bc-41ac-b3fb-f326f3973105" containerID="f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd" exitCode=137
Jul 10 15:16:03.572672 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.572178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt" event={"ID":"2a0cad14-49bc-41ac-b3fb-f326f3973105","Type":"ContainerDied","Data":"f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd"}
Jul 10 15:16:03.572672 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.572245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt" event={"ID":"2a0cad14-49bc-41ac-b3fb-f326f3973105","Type":"ContainerDied","Data":"4819f01d38f4b6fc2e6f7de711fc49223dd20ea97cbfb281fc692164c5e84287"}
Jul 10 15:16:03.572672 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.572317    2583 scope.go:117] "RemoveContainer" containerID="f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd"
Jul 10 15:16:03.572859 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.572695    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"
Jul 10 15:16:03.599753 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.599005    2583 scope.go:117] "RemoveContainer" containerID="f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd"
Jul 10 15:16:03.599927 ip-10-0-136-210 kubenswrapper[2583]: E0710 15:16:03.599897    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd\": container with ID starting with f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd not found: ID does not exist" containerID="f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd"
Jul 10 15:16:03.600021 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.599939    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd"} err="failed to get container status \"f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd\": rpc error: code = NotFound desc = could not find container \"f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd\": container with ID starting with f018abcd337f60bb50cf292dac88a3bd4020e9996380187ff092d295773defdd not found: ID does not exist"
Jul 10 15:16:03.614006 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.613975    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"]
Jul 10 15:16:03.614439 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:03.614420    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e538f1-29728275-frkvt"]
Jul 10 15:16:04.754642 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.754610    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/alertmanager/0.log"
Jul 10 15:16:04.783775 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.783744    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/config-reloader/0.log"
Jul 10 15:16:04.810231 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.810201    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/kube-rbac-proxy-web/0.log"
Jul 10 15:16:04.839726 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.839699    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/kube-rbac-proxy/0.log"
Jul 10 15:16:04.850228 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.850194    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a0cad14-49bc-41ac-b3fb-f326f3973105" path="/var/lib/kubelet/pods/2a0cad14-49bc-41ac-b3fb-f326f3973105/volumes"
Jul 10 15:16:04.864378 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.864351    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/kube-rbac-proxy-metric/0.log"
Jul 10 15:16:04.889728 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.889684    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/prom-label-proxy/0.log"
Jul 10 15:16:04.913422 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:04.913392    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a59f7884-efb6-49ec-b636-a0ee313cc909/init-config-reloader/0.log"
Jul 10 15:16:05.103562 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.103533    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-chfpp_06148697-3ebc-41b0-a129-6e073dfa1bb3/monitoring-plugin/0.log"
Jul 10 15:16:05.229093 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.229054    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n4sc5_ae37bf2b-3bb1-4fe8-94ef-8651a17db324/node-exporter/0.log"
Jul 10 15:16:05.251323 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.251297    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n4sc5_ae37bf2b-3bb1-4fe8-94ef-8651a17db324/kube-rbac-proxy/0.log"
Jul 10 15:16:05.278230 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.278082    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n4sc5_ae37bf2b-3bb1-4fe8-94ef-8651a17db324/init-textfile/0.log"
Jul 10 15:16:05.377598 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.377568    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wmr2x_ab466a2b-183b-42e0-b372-edc6982a946d/kube-rbac-proxy-main/0.log"
Jul 10 15:16:05.400854 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.400827    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wmr2x_ab466a2b-183b-42e0-b372-edc6982a946d/kube-rbac-proxy-self/0.log"
Jul 10 15:16:05.421663 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.421631    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wmr2x_ab466a2b-183b-42e0-b372-edc6982a946d/openshift-state-metrics/0.log"
Jul 10 15:16:05.477959 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.477926    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/prometheus/0.log"
Jul 10 15:16:05.504732 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.504698    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/config-reloader/0.log"
Jul 10 15:16:05.527004 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.526965    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/thanos-sidecar/0.log"
Jul 10 15:16:05.557592 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.557298    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/kube-rbac-proxy-web/0.log"
Jul 10 15:16:05.590773 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.590743    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/kube-rbac-proxy/0.log"
Jul 10 15:16:05.615629 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.615600    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/kube-rbac-proxy-thanos/0.log"
Jul 10 15:16:05.638701 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.638668    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4cf1b68-d3e6-42ca-9067-33bdbd191917/init-config-reloader/0.log"
Jul 10 15:16:05.672140 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.672109    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-l87jd_c0028d35-0a41-4d17-a319-ed2d4cee9168/prometheus-operator/0.log"
Jul 10 15:16:05.693491 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.693459    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-l87jd_c0028d35-0a41-4d17-a319-ed2d4cee9168/kube-rbac-proxy/0.log"
Jul 10 15:16:05.721835 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:05.721802    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-nv645_fcd96d02-8d9c-436e-aa44-a7a06b4a049d/prometheus-operator-admission-webhook/0.log"
Jul 10 15:16:07.156449 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.156411    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-bm7wb_08c26ab0-91eb-441c-9a0b-858c6fb2fa43/networking-console-plugin/0.log"
Jul 10 15:16:07.584857 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.584826    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"]
Jul 10 15:16:07.585501 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.585477    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a0cad14-49bc-41ac-b3fb-f326f3973105" containerName="cleanup"
Jul 10 15:16:07.585650 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.585548    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a0cad14-49bc-41ac-b3fb-f326f3973105" containerName="cleanup"
Jul 10 15:16:07.585761 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.585745    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a0cad14-49bc-41ac-b3fb-f326f3973105" containerName="cleanup"
Jul 10 15:16:07.594784 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.594757    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.603591 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.603566    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"]
Jul 10 15:16:07.675759 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.675723    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-sys\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.675947 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.675917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-lib-modules\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.676028 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.675988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj7n4\" (UniqueName: \"kubernetes.io/projected/6f36d112-4219-46c4-a599-53fb94f82c66-kube-api-access-xj7n4\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.676087 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.676028    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-podres\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.676141 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.676103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-proc\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777261 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777225    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-proc\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777444 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777283    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-sys\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777444 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-lib-modules\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777444 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-proc\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777444 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj7n4\" (UniqueName: \"kubernetes.io/projected/6f36d112-4219-46c4-a599-53fb94f82c66-kube-api-access-xj7n4\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777444 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-podres\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777444 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777422    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-sys\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777709 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777468    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-lib-modules\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.777709 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.777555    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f36d112-4219-46c4-a599-53fb94f82c66-podres\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.787229 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.787178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj7n4\" (UniqueName: \"kubernetes.io/projected/6f36d112-4219-46c4-a599-53fb94f82c66-kube-api-access-xj7n4\") pod \"perf-node-gather-daemonset-nzl66\" (UID: \"6f36d112-4219-46c4-a599-53fb94f82c66\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:07.913964 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:07.913924    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:08.098936 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:08.098906    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"]
Jul 10 15:16:08.101894 ip-10-0-136-210 kubenswrapper[2583]: W0710 15:16:08.101860    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f36d112_4219_46c4_a599_53fb94f82c66.slice/crio-cd97c69c1cdc6acf9a9a6bd547183decb1b7090dc4d22f5e413796fa86903010 WatchSource:0}: Error finding container cd97c69c1cdc6acf9a9a6bd547183decb1b7090dc4d22f5e413796fa86903010: Status 404 returned error can't find the container with id cd97c69c1cdc6acf9a9a6bd547183decb1b7090dc4d22f5e413796fa86903010
Jul 10 15:16:08.518003 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:08.517911    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-wldc5_801e38db-10ba-44b5-90c4-bcce65f80b3f/download-server/0.log"
Jul 10 15:16:08.616850 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:08.616024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66" event={"ID":"6f36d112-4219-46c4-a599-53fb94f82c66","Type":"ContainerStarted","Data":"7ef652f313f513f644e3cb9e5d32493e7a677aff0c0f658dd7b5b67becc77266"}
Jul 10 15:16:08.616850 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:08.616092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66" event={"ID":"6f36d112-4219-46c4-a599-53fb94f82c66","Type":"ContainerStarted","Data":"cd97c69c1cdc6acf9a9a6bd547183decb1b7090dc4d22f5e413796fa86903010"}
Jul 10 15:16:08.617084 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:08.616927    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:08.634464 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:08.634405    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66" podStartSLOduration=1.634387495 podStartE2EDuration="1.634387495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:16:08.633843481 +0000 UTC m=+2466.340164952" watchObservedRunningTime="2026-07-10 15:16:08.634387495 +0000 UTC m=+2466.340708966"
Jul 10 15:16:10.256351 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:10.256319    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vdw48_92643ff5-ab30-43fd-8106-4abb0940b6cd/dns/0.log"
Jul 10 15:16:10.277185 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:10.277153    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vdw48_92643ff5-ab30-43fd-8106-4abb0940b6cd/kube-rbac-proxy/0.log"
Jul 10 15:16:10.301136 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:10.301110    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-prtg5_5944324d-e133-48a0-920c-26ae31508a68/dns-node-resolver/0.log"
Jul 10 15:16:10.917851 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:10.917813    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-474nb_3055e9b1-3af7-4d56-83ca-574b55464161/node-ca/0.log"
Jul 10 15:16:12.203165 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:12.203127    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-p2dhc_0186c81d-ae51-4005-81fe-ca87633d1494/kube-auth-proxy/0.log"
Jul 10 15:16:12.240285 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:12.240249    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/0.log"
Jul 10 15:16:12.302100 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:12.302061    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/1.log"
Jul 10 15:16:12.346315 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:12.346274    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gwjx9_b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f/payload-pre-processing/0.log"
Jul 10 15:16:13.108178 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:13.108147    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-82rgq_3a6052cf-119f-45ad-afe3-ac6e322a8433/serve-healthcheck-canary/0.log"
Jul 10 15:16:13.695249 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:13.695221    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rfr2g_657b971b-c38c-45c7-a2e8-f47c4c2f0f68/kube-rbac-proxy/0.log"
Jul 10 15:16:13.719870 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:13.719841    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rfr2g_657b971b-c38c-45c7-a2e8-f47c4c2f0f68/exporter/0.log"
Jul 10 15:16:13.744511 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:13.744483    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rfr2g_657b971b-c38c-45c7-a2e8-f47c4c2f0f68/extractor/0.log"
Jul 10 15:16:15.640436 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:15.640407    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-nzl66"
Jul 10 15:16:16.146552 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.146496    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mq8jn_be021526-cbe6-4f16-a1ac-50d9c38b12ac/manager/0.log"
Jul 10 15:16:16.383604 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.383573    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dl6t6_f23f7574-ffe1-474e-b36c-617f367240b0/manager/0.log"
Jul 10 15:16:16.539879 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.539841    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b89cd5547-8cpt7_cd521ab1-8096-47e5-91f9-782408cf0dc2/manager/0.log"
Jul 10 15:16:16.566895 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.566870    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rqg5c_b85bd97c-19c6-45ea-9fa7-3505181a3293/server/0.log"
Jul 10 15:16:16.685947 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.685909    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zbx64_51a6fdb3-a4da-4ac3-b8b3-5f234ade7048/manager/0.log"
Jul 10 15:16:16.714471 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.714440    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-8xmv4_e9febbf1-8597-481a-84f2-c344b636cd41/manager/0.log"
Jul 10 15:16:16.738157 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:16.738127    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-8xmv4_e9febbf1-8597-481a-84f2-c344b636cd41/copy-manifests/0.log"
Jul 10 15:16:18.115373 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:18.115345    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-p2dhc_0186c81d-ae51-4005-81fe-ca87633d1494/kube-auth-proxy/0.log"
Jul 10 15:16:18.136809 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:18.136767    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/0.log"
Jul 10 15:16:18.191061 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:18.190965    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/1.log"
Jul 10 15:16:18.220198 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:18.220169    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gwjx9_b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f/payload-pre-processing/0.log"
Jul 10 15:16:18.362543 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:18.362484    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-k62hf_5c319c34-989c-4ea8-b7ce-216cc73f799e/openshift-lws-operator/0.log"
Jul 10 15:16:19.731405 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:19.731330    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-p2dhc_0186c81d-ae51-4005-81fe-ca87633d1494/kube-auth-proxy/0.log"
Jul 10 15:16:19.757983 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:19.757900    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/0.log"
Jul 10 15:16:19.821423 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:19.821388    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-pzt5b_e0f16b21-14c2-4081-b5a8-6699f5840a7b/istio-proxy/1.log"
Jul 10 15:16:19.848966 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:19.848937    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gwjx9_b98c6a9a-9a25-48eb-a5a8-33e0ea580c2f/payload-pre-processing/0.log"
Jul 10 15:16:24.998235 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:24.998205    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/kube-multus-additional-cni-plugins/0.log"
Jul 10 15:16:25.022256 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:25.022235    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/egress-router-binary-copy/0.log"
Jul 10 15:16:25.046727 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:25.046701    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/cni-plugins/0.log"
Jul 10 15:16:25.071679 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:25.071644    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/bond-cni-plugin/0.log"
Jul 10 15:16:25.099992 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:25.099965    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/routeoverride-cni/0.log"
Jul 10 15:16:25.124658 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:25.124631    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/whereabouts-cni-bincopy/0.log"
Jul 10 15:16:25.153871 ip-10-0-136-210 kubenswrapper[2583]: I0710 15:16:25.153841    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqlw9_44846e01-1b2b-453b-ab4a-68fd1b2a28e7/whereabouts-cni/0.log"