Jul 07 13:25:32.288013 ip-10-0-129-217 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 13:25:32.288023 ip-10-0-129-217 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 13:25:32.288030 ip-10-0-129-217 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 13:25:32.288234 ip-10-0-129-217 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 13:25:42.406409 ip-10-0-129-217 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 13:25:42.406423 ip-10-0-129-217 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8e15e0897cae4f3f9e5f35197b0e910f --
Jul 07 13:27:45.321538 ip-10-0-129-217 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 13:27:45.803494 ip-10-0-129-217 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 13:27:45.803494 ip-10-0-129-217 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 13:27:45.803494 ip-10-0-129-217 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 13:27:45.803494 ip-10-0-129-217 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 13:27:45.803494 ip-10-0-129-217 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 13:27:45.805216 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.805118    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 13:27:45.811442 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811426    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:27:45.811442 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811441    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811446    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811450    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811453    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811457    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811460    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811463    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811466    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811469    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811472    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811475    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811478    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811481    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811484    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811486    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811489    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811492    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811494    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811498    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811501    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:27:45.811507 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811503    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811506    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811509    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811512    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811515    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811518    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811522    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811525    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811530    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811535    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811538    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811541    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811544    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811546    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811549    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811552    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811555    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811557    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811560    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:27:45.812020 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811563    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811565    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811568    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811571    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811573    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811576    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811579    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811582    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811584    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811587    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811589    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811592    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811595    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811597    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811600    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811603    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811606    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811609    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811612    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811615    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:27:45.812521 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811618    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811620    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811623    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811627    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811629    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811632    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811635    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811638    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811640    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811643    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811646    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811650    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811654    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811657    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811660    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811664    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811666    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811669    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811672    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:27:45.813048 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811675    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811677    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811680    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811683    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811686    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811690    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811693    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811696    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811700    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811703    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811708    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811712    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811715    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811718    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811721    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811724    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811727    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811730    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811733    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:27:45.813528 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.811736    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813606    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813615    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813618    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813621    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813624    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813627    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813632    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813636    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813639    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813642    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813646    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813649    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813652    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813654    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813657    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813660    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813662    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813665    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813668    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:27:45.814018 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813671    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813674    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813677    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813679    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813682    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813684    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813687    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813689    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813692    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813695    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813698    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813701    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813704    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813708    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813713    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813716    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813720    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813723    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813726    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:27:45.814817 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813728    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813731    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813734    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813737    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813739    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813742    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813744    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813747    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813750    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813752    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813755    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813757    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813760    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813764    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813767    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813769    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813772    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813774    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813777    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:27:45.815350 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813780    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813783    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813786    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813788    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813791    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813794    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813797    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813800    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813803    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813806    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813809    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813813    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813817    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813838    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813842    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813845    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813849    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813854    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813858    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813862    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:27:45.816044 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813866    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813869    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813873    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813876    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813879    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813882    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813884    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813887    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813890    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813893    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813895    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813898    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813901    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813903    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813906    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813909    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813912    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813915    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813917    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813922    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:27:45.816992 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813926    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.813929    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816290    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816306    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816312    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816317    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816322    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816327    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816331    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816337    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816342    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816347    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816351    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816356    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816361    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816365    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816370    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816374    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816378    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816383    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:27:45.817925 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816388    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816392    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816397    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816401    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816407    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816413    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816418    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816422    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816426    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816431    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816435    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816439    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816443    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816448    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816459    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816465    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816470    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816476    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816480    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:27:45.818711 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816485    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816490    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816495    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816499    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816504    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816508    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816513    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816520    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816527    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816532    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816537    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816544    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816549    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816556    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816561    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816567    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816571    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816576    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816581    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:27:45.819298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816586    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816591    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816596    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816601    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816606    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816610    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816615    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816619    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816624    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816630    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816634    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816639    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816643    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816648    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816652    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816656    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816660    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816665    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816670    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816675    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:27:45.819910 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816679    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816684    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816688    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816693    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816698    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816702    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816707    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816712    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816716    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816721    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816726    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816730    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816735    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816739    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816743    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816748    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816752    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816756    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816761    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:27:45.820678 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816766    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816770    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816799    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.816807    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816931    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816943    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816955    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816967    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816974    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816980    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816987    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816993    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.816998    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817003    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817009    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817014    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817019    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817024    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817029    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817034    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817040    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817046    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817053    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 07 13:27:45.821390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817059    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817064    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817069    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817075    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817082    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817087    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817092    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817097    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817103    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817108    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817113    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817120    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817125    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817132    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817138    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817143    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817148    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817153    2582 flags.go:64] FLAG: --enable-server="true"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817157    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817165    2582 flags.go:64] FLAG: --event-burst="100"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817170    2582 flags.go:64] FLAG: --event-qps="50"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817175    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817180    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817186    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 07 13:27:45.822018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817193    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817197    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817202    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817208    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817213    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817219    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817224    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817230    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817235    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817240    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817244    2582 flags.go:64] FLAG: --feature-gates=""
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817251    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817256    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817262    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817267    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817273    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817278    2582 flags.go:64] FLAG: --help="false"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817284    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-129-217.ec2.internal"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817289    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817295    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817301    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817307    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817314    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817319    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 13:27:45.822738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817324    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817329    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817334    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817339    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817344    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817349    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817354    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817359    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817364    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817369    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817374    2582 flags.go:64] FLAG: --lock-file=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817378    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817384    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817389    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817399    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817404    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817410    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817415    2582 flags.go:64] FLAG: --logging-format="text"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817420    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817426    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817431    2582 flags.go:64] FLAG: --manifest-url=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817436    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817443    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817449    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817456    2582 flags.go:64] FLAG: --max-pods="110"
Jul 07 13:27:45.823438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817461    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817466    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817471    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817476    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817481    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817486    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817491    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817503    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817508    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817513    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817519    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817524    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817534    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817539    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817544    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817549    2582 flags.go:64] FLAG: --port="10250"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817554    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817559    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0afeaf811ca67a881"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817565    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817570    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817576    2582 flags.go:64] FLAG: --register-node="true"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817580    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817586    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817591    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 07 13:27:45.824148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817597    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817603    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817609    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817614    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817619    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817624    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817629    2582 flags.go:64] FLAG: --runonce="false"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817633    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817639    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817644    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817649    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817654    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817660    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817665    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817670    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817675    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817680    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817685    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817690    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817696    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817701    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817706    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817716    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817721    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817725    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 13:27:45.824768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817733    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817738    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817743    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817748    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817753    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817757    2582 flags.go:64] FLAG: --v="2"
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817764    2582 flags.go:64] FLAG: --version="false"
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817771    2582 flags.go:64] FLAG: --vmodule=""
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817778    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.817784    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.817995    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818005    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818011    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818016    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818021    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818025    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818029    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818038    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818043    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818048    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818053    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:27:45.825500 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818058    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818062    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818067    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818072    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818076    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818081    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818085    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818090    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818094    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818099    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818103    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818108    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818112    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818117    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818121    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818126    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818130    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818135    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818139    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818144    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:27:45.826147 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818150    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818154    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818159    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818164    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818168    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818175    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818182    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818189    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818197    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818203    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818209    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818214    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818219    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818224    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818230    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818234    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818241    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818246    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818251    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818255    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:27:45.826657 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818260    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818264    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818269    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818274    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818279    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818283    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818287    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818292    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818297    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818301    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818306    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818310    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818314    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818320    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818324    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818329    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818333    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818338    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818342    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818347    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:27:45.827292 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818353    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818357    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818362    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818366    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818374    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818380    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818385    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818390    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818397    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818402    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818406    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818411    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818415    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818420    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818424    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818429    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818433    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818437    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818442    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:27:45.827810 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818446    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818450    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818456    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818461    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818465    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818469    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818474    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818479    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.818483    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.819494    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.827383    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.827398    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.827419    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.827426    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827485    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:27:45.828340 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827491    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827496    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827500    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827503    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827507    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827510    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827513    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827516    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827520    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827522    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827525    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827528    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827531    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827534    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827537    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827539    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827543    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827545    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827548    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:27:45.828764 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827551    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827555    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827558    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827561    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827564    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827568    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827571    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827573    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827576    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827580    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827583    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827587    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827590    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827592    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827595    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827598    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827601    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827604    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827606    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:27:45.829262 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827609    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827612    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827615    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827618    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827621    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827623    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827626    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827629    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827632    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827636    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827640    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827642    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827645    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827648    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827651    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827654    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827657    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827660    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827664    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:27:45.829744 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827668    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827672    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827675    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827679    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827683    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827686    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827689    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827692    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827694    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827697    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827700    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827703    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827705    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827708    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827711    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827714    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827717    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827719    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827722    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827725    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:27:45.830239 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827728    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827731    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827733    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827736    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827739    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827741    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827744    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827748    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827750    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827753    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827756    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827759    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827762    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827764    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827767    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827770    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827774    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827777    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827780    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827782    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:27:45.830752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827785    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.827790    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827926    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827933    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827936    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827939    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827942    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827945    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827948    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827951    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827954    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827957    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827959    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827962    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827965    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:27:45.831362 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827968    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827970    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827973    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827976    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827978    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827982    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827985    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827987    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.827999    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828002    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828005    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828008    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828011    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828014    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828017    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828021    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828024    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828027    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828030    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828032    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:27:45.831752 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828035    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828038    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828041    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828044    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828046    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828049    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828052    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828054    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828057    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828060    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828062    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828065    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828068    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828070    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828073    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828076    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828079    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828083    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828085    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828088    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:27:45.832284 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828091    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828095    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828099    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828102    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828105    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828109    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828112    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828115    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828120    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828123    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828126    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828129    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828132    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828136    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828140    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828143    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828146    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828149    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828151    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:27:45.832803 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828154    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828156    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828159    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828162    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828165    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828167    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828170    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828172    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828175    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828178    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828181    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828183    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828186    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828189    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828192    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828195    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828197    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828200    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828203    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828206    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:27:45.833303 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828209    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828211    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828215    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828217    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828220    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828223    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:45.828225    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.828231    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.828343    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.830637    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.832261    2582 server.go:1019] "Starting client certificate rotation"
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.832364    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 13:27:45.833820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.832404    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 13:27:45.835339 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.835325    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 13:27:45.836695 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.836682    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 13:27:45.848872 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.848754    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 07 13:27:45.852143 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.852127    2582 log.go:25] "Validated CRI v1 image API"
Jul 07 13:27:45.853040 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.853017    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 13:27:45.856634 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.856614    2582 fs.go:135] Filesystem UUIDs: map[121c4438-a735-4114-a1b8-1ae592971f72:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b3534987-aac1-4a58-99a1-a51d87431fb1:/dev/nvme0n1p3]
Jul 07 13:27:45.856703 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.856633    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 13:27:45.862588 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.862475    2582 manager.go:216] Machine: {Timestamp:2026-07-07 13:27:45.861302067 +0000 UTC m=+0.394207803 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099989 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21dd6accfc2fd533bb73a1776f27b1 SystemUUID:ec21dd6a-ccfc-2fd5-33bb-73a1776f27b1 BootID:8e15e089-7cae-4f3f-9e5f-35197b0e910f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:22:0f:a7:4b:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:22:0f:a7:4b:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:f5:86:5e:50:41 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 13:27:45.862588 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.862582    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 13:27:45.862714 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.862702    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 13:27:45.864530 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.864508    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 13:27:45.864668 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.864532    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-217.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 13:27:45.865325 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.865314    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 13:27:45.865357 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.865326    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 13:27:45.865357 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.865334    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 13:27:45.866264 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.866253    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 13:27:45.866564 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.866549    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 13:27:45.867907 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.867895    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 13:27:45.871370 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871358    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 13:27:45.871476 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871467    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 13:27:45.871515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871496    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 13:27:45.871515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871506    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 13:27:45.871572 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871520    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 13:27:45.871572 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871530    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 07 13:27:45.871572 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871539    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 13:27:45.871930 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871906    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 13:27:45.872021 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.871947    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 13:27:45.874389 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.874358    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 13:27:45.876502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.876487    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 13:27:45.876586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.876519    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 13:27:45.877723 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877711    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877727    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877734    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877740    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877749    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877755    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877763    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877769    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877776    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877781    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877790    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 13:27:45.877800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.877796    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 13:27:45.879306 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.879292    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 13:27:45.879306 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.879305    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 13:27:45.881656 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.881634    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 13:27:45.881656 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.881650    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 13:27:45.881767 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.881754    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-217.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 13:27:45.882737 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.882723    2582 server.go:1268] "Started kubelet"
Jul 07 13:27:45.882893 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.882839    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 13:27:45.882954 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.882929    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 13:27:45.883788 ip-10-0-129-217 systemd[1]: Started Kubernetes Kubelet.
Jul 07 13:27:45.884014 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.883968    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 13:27:45.886678 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.886661    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 13:27:45.887732 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.887707    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 13:27:45.889540 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.889518    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 13:27:45.893977 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.893956    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 13:27:45.893977 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.893970    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 13:27:45.894132 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.894105    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:45.894640 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.894621    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 13:27:45.894749 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.894729    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 13:27:45.894816 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.894753    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 13:27:45.894958 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.894940    2582 factory.go:55] Registering systemd factory
Jul 07 13:27:45.895027 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.894962    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 07 13:27:45.895191 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895178    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 13:27:45.895191 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895190    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 13:27:45.895322 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895200    2582 factory.go:153] Registering CRI-O factory
Jul 07 13:27:45.895322 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895217    2582 factory.go:223] Registration of the crio container factory successfully
Jul 07 13:27:45.895322 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895265    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 13:27:45.895322 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895289    2582 factory.go:103] Registering Raw factory
Jul 07 13:27:45.895322 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895305    2582 manager.go:1196] Started watching for new ooms in manager
Jul 07 13:27:45.895756 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.895742    2582 manager.go:349] Starting recovery of all containers
Jul 07 13:27:45.897869 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.897832    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4dp6x"
Jul 07 13:27:45.903331 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.903287    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 13:27:45.903478 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.903437    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 13:27:45.903607 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.903558    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 13:27:45.904371 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.903053    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18c004ec6a67b24d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-07-07 13:27:45.882690125 +0000 UTC m=+0.415595845,LastTimestamp:2026-07-07 13:27:45.882690125 +0000 UTC m=+0.415595845,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}"
Jul 07 13:27:45.904479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.904447    2582 manager.go:354] Recovery completed
Jul 07 13:27:45.909006 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.908984    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4dp6x"
Jul 07 13:27:45.910216 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.910203    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:45.912040 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912025    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:45.912088 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912057    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:45.912088 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912074    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:45.912613 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912598    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 13:27:45.912613 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912609    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 13:27:45.912710 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912615    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:45.912710 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.912624    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 13:27:45.913307 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.913291    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:45.913358 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.913317    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:45.913358 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.913327    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:45.913953 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.913941    2582 policy_none.go:49] "None policy: Start"
Jul 07 13:27:45.913991 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.913958    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 13:27:45.913991 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.913968    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 13:27:45.914356 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.914295    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18c004ec6c2781fe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-217.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-07-07 13:27:45.912037886 +0000 UTC m=+0.444943605,LastTimestamp:2026-07-07 13:27:45.912037886 +0000 UTC m=+0.444943605,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}"
Jul 07 13:27:45.915542 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.915528    2582 policy_none.go:47] "Start"
Jul 07 13:27:45.952458 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952439    2582 manager.go:337] "Starting Device Plugin manager"
Jul 07 13:27:45.952564 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.952475    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 13:27:45.952564 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952486    2582 server.go:87] "Starting device plugin registration server"
Jul 07 13:27:45.952749 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952736    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 13:27:45.952785 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952757    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 13:27:45.952895 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952872    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 13:27:45.952988 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952958    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 13:27:45.952988 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:45.952967    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 13:27:45.953710 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.953511    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 13:27:45.953776 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:45.953750    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.053411 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.053372    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:46.054586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.054409    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:46.054586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.054445    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:46.054586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.054461    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:46.054586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.054488    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.064038 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.064018    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.064106 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.064040    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-217.ec2.internal\": node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.072039 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.072019    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 13:27:46.072157 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.072070    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 13:27:46.072157 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.072093    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 13:27:46.072157 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.072131    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 13:27:46.075104 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.075084    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:27:46.081889 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.081869    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.174653 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.174615    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"]
Jul 07 13:27:46.174841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.174704    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:46.177789 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.177769    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:46.177882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.177812    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:46.177882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.177825    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:46.179303 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.179289    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:46.179431 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.179407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.179531 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.179449    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:46.180160 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.180144    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:46.180235 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.180174    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:46.180235 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.180174    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:46.180235 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.180209    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:46.180235 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.180224    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:46.180427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.180188    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:46.181683 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.181665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.181775 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.181699    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:27:46.182027 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.182011    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.182439 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.182424    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:27:46.182511 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.182456    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:27:46.182511 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.182472    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:27:46.197423 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.197406    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c857f80ddb13de0ede08c410a7a78ba-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"1c857f80ddb13de0ede08c410a7a78ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.197490 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.197431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.197490 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.197447    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.200438 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.200423    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.204868 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.204834    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.282460 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.282426    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.298118 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.298094    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c857f80ddb13de0ede08c410a7a78ba-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"1c857f80ddb13de0ede08c410a7a78ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.298200 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.298122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.298200 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.298141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.298200 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.298188    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.298329 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.298225    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.298329 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.298227    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c857f80ddb13de0ede08c410a7a78ba-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"1c857f80ddb13de0ede08c410a7a78ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.383169 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.383092    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.489276 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.489229    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.503767 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.503742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.508411 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.508258    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:46.592956 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.592918    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.700662 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.700577    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.801834 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.801792    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.832074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.832026    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 13:27:46.832511 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.832200    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 13:27:46.902615 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:46.902584    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:46.912006 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.911968    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 13:22:46 +0000 UTC" deadline="2027-12-17 18:06:33.938511238 +0000 UTC"
Jul 07 13:27:46.912006 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:46.912002    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12676h38m47.02651274s"
Jul 07 13:27:47.003128 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:47.003098    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:47.053056 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:47.053021    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c857f80ddb13de0ede08c410a7a78ba.slice/crio-1f148eb31a69366d73504e824e21e10a6a1c2fe51be4e32b5cdff1016e9db5a4 WatchSource:0}: Error finding container 1f148eb31a69366d73504e824e21e10a6a1c2fe51be4e32b5cdff1016e9db5a4: Status 404 returned error can't find the container with id 1f148eb31a69366d73504e824e21e10a6a1c2fe51be4e32b5cdff1016e9db5a4
Jul 07 13:27:47.060715 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.060700    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:27:47.075740 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.075700    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" event={"ID":"1c857f80ddb13de0ede08c410a7a78ba","Type":"ContainerStarted","Data":"1f148eb31a69366d73504e824e21e10a6a1c2fe51be4e32b5cdff1016e9db5a4"}
Jul 07 13:27:47.103717 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:47.103691    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found"
Jul 07 13:27:47.116834 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:47.116808    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dad3d90ea1a5450d51ca0838b0d656.slice/crio-d95bc11cef0359c8e98b3bffa670ecc8c119e5b2af03f59e9dd1a8235e80df7c WatchSource:0}: Error finding container d95bc11cef0359c8e98b3bffa670ecc8c119e5b2af03f59e9dd1a8235e80df7c: Status 404 returned error can't find the container with id d95bc11cef0359c8e98b3bffa670ecc8c119e5b2af03f59e9dd1a8235e80df7c
Jul 07 13:27:47.159992 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.159969    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:27:47.179025 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.179005    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:27:47.194328 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.194307    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:47.208294 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.208231    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 13:27:47.209386 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.209371    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"
Jul 07 13:27:47.221897 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.221870    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 13:27:47.493152 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.493070    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:27:47.872968 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.872804    2582 apiserver.go:52] "Watching apiserver"
Jul 07 13:27:47.882573 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.882539    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:27:47.884119 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.884093    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 13:27:47.886385 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.886360    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-pjn5s","kube-system/konnectivity-agent-48bvw","kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222","openshift-cluster-node-tuning-operator/tuned-5qvr8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal","openshift-multus/network-metrics-daemon-sg4vv","openshift-network-diagnostics/network-check-target-ttjh4","openshift-dns/node-resolver-5jqwk","openshift-image-registry/node-ca-xhntt","openshift-multus/multus-additional-cni-plugins-7r4gz","openshift-multus/multus-vh9cj","openshift-network-operator/iptables-alerter-xm8rd","openshift-ovn-kubernetes/ovnkube-node-xmv42"]
Jul 07 13:27:47.888688 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.888668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:47.888790 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:47.888749    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:27:47.889838 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.889818    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:47.892854 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.892822    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.896320 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.896298    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 13:27:47.896873 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.896834    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.897093 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.897048    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v2jwt\""
Jul 07 13:27:47.897179 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.897111    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.897311 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.897288    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 13:27:47.897414 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.897394    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.898114 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.898093    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 13:27:47.898252 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.898125    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qlnhp\""
Jul 07 13:27:47.898252 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.898148    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.898252 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.898096    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 13:27:47.898421 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.898324    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:47.898733 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.898713    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.900759 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.900660    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:47.900759 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:47.900724    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:27:47.902094 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.901991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:47.903446 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.903423    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:47.903549 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:47.903493    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:27:47.904461 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.904440    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 13:27:47.904586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.904431    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 13:27:47.904709 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.904692    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.906378 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.905746    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:47.906502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.906476    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.906758 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.906739    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.907165 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907148    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 13:27:47.907407 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907389    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-84fpw\""
Jul 07 13:27:47.907630 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907614    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.907716 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907658    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-ovn\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.907716 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-cni-netd\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.907826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.907826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovnkube-config\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.907826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-etc-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.907826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907807    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-registration-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907835    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhmzt\" (UniqueName: \"kubernetes.io/projected/e0959543-1c8b-419f-b476-1d92e3c168d0-kube-api-access-zhmzt\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-node-log\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovnkube-script-lib\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-sys-fs\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907966    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p67xp\""
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.907971    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-kubelet\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-slash\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908053    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-socket-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908079    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovn-node-metrics-cert\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0959543-1c8b-419f-b476-1d92e3c168d0-iptables-alerter-script\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908224    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-cni-bin\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-device-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2pcf\" (UniqueName: \"kubernetes.io/projected/dc93d94a-3701-44a8-8274-ac78bdca6b3f-kube-api-access-h2pcf\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-run-netns\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908367    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-var-lib-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-log-socket\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908418    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzkvp\" (UniqueName: \"kubernetes.io/projected/58d8012a-5618-4d1d-aa6f-bfae87401dfc-kube-api-access-bzkvp\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.908499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908443    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-etc-selinux\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jkwc\" (UniqueName: \"kubernetes.io/projected/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-kube-api-access-9jkwc\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-env-overrides\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0959543-1c8b-419f-b476-1d92e3c168d0-host-slash\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908953    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-systemd-units\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-systemd\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:47.909113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.908995    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.909502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.909131    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ltq29\""
Jul 07 13:27:47.909872 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.909718    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 13:27:47.910347 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.910098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:47.912001 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.911980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:47.914175 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.914155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:47.922018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.922000    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.922118 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.922074    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.922164 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.922118    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 13:27:47.928094 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.928072    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 13:27:47.928738 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.928688    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 13:27:47.929259 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.929238    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 13:27:47.930672 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.930651    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 13:27:47.934157 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.934139    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-72wt6\""
Jul 07 13:27:47.934510 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.934490    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 13:27:47.934759 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.934743    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xxlr6\""
Jul 07 13:27:47.934759 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.934750    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 13:27:47.935070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.935050    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6ld8h\""
Jul 07 13:27:47.935148 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.935050    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-42sgz\""
Jul 07 13:27:47.938035 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.938017    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 13:27:47.938131 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.938046    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 13:27:47.942543 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.942520    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w4fh5"
Jul 07 13:27:47.960628 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.960606    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w4fh5"
Jul 07 13:27:47.995830 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:47.995800    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 13:27:48.010221 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-systemd\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.010372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-host\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.010372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-os-release\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.010372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prvwn\" (UniqueName: \"kubernetes.io/projected/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-kube-api-access-prvwn\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.010372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010309    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-cni-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.010372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010342    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-cnibin\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.010372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010368    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-socket-dir-parent\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-kubelet\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-kubelet\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-socket-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010475    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-lib-modules\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2x9t\" (UniqueName: \"kubernetes.io/projected/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-kube-api-access-q2x9t\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010528    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010553    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-system-cni-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010580    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovn-node-metrics-cert\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-etc-kubernetes\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0959543-1c8b-419f-b476-1d92e3c168d0-iptables-alerter-script\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.010687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010659    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-cni-bin\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010690    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-device-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2pcf\" (UniqueName: \"kubernetes.io/projected/dc93d94a-3701-44a8-8274-ac78bdca6b3f-kube-api-access-h2pcf\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-kubernetes\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cnibin\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-run-netns\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-log-socket\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010912    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-run-netns\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010917    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-kubelet\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.010994    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-cni-bin\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011067    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-log-socket\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011100    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-device-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jkwc\" (UniqueName: \"kubernetes.io/projected/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-kube-api-access-9jkwc\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011148    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011181    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-modprobe-d\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011199    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-socket-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.011249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011226    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87706558-76f0-4fbe-88d8-69c886cdf9e0-tmp\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlgtj\" (UniqueName: \"kubernetes.io/projected/87706558-76f0-4fbe-88d8-69c886cdf9e0-kube-api-access-zlgtj\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011406    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8db6f404-a4ed-47b3-9850-eb6b8b776e74-hosts-file\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-env-overrides\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011483    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-host\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011531    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0959543-1c8b-419f-b476-1d92e3c168d0-iptables-alerter-script\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db6f404-a4ed-47b3-9850-eb6b8b776e74-tmp-dir\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-multus-certs\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011696    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0959543-1c8b-419f-b476-1d92e3c168d0-host-slash\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-systemd-units\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011761    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-ovn\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011826    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0959543-1c8b-419f-b476-1d92e3c168d0-host-slash\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-cni-netd\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011886    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-ovn\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-system-cni-dir\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011930    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-systemd-units\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011933    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-env-overrides\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-registration-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011972    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-cni-netd\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.011984    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f8332404-cb66-4442-a5e4-59b4ea994ab7-kubelet-config\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012017    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-registration-dir\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-run\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cni-binary-copy\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhmzt\" (UniqueName: \"kubernetes.io/projected/e0959543-1c8b-419f-b476-1d92e3c168d0-kube-api-access-zhmzt\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-node-log\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovnkube-script-lib\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-sys-fs\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-var-lib-kubelet\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012221    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-node-log\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012243    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-serviceca\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-k8s-cni-cncf-io\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.012801 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012315    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-cni-multus\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012332    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-sys-fs\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-slash\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012411    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012447    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-slash\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-sys\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da30c936-4b65-4167-8891-2b6efc601ccd-cni-binary-copy\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-hostroot\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f8332404-cb66-4442-a5e4-59b4ea994ab7-dbus\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c112cf11-8255-4826-906e-0b8a815f125b-agent-certs\") pod \"konnectivity-agent-48bvw\" (UID: \"c112cf11-8255-4826-906e-0b8a815f125b\") " pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.012647    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysconfig\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxc7j\" (UniqueName: \"kubernetes.io/projected/da30c936-4b65-4167-8891-2b6efc601ccd-kube-api-access-pxc7j\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012710    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.012736    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:48.512703384 +0000 UTC m=+3.045609107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:48.013614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012750    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012764    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012812    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovnkube-script-lib\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012824    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-tuned\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012891    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-var-lib-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012966    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-var-lib-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.012977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzkvp\" (UniqueName: \"kubernetes.io/projected/58d8012a-5618-4d1d-aa6f-bfae87401dfc-kube-api-access-bzkvp\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-etc-selinux\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013044    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-conf-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013068    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysctl-conf\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013093    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-netns\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-systemd\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013201    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-run-systemd\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013214    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc93d94a-3701-44a8-8274-ac78bdca6b3f-etc-selinux\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovnkube-config\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.014513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c112cf11-8255-4826-906e-0b8a815f125b-konnectivity-ca\") pod \"konnectivity-agent-48bvw\" (UID: \"c112cf11-8255-4826-906e-0b8a815f125b\") " pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013336    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysctl-d\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013383    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7xkj\" (UniqueName: \"kubernetes.io/projected/8db6f404-a4ed-47b3-9850-eb6b8b776e74-kube-api-access-n7xkj\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013418    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-os-release\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-etc-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-cni-bin\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013517    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58d8012a-5618-4d1d-aa6f-bfae87401dfc-etc-openvswitch\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013551    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/da30c936-4b65-4167-8891-2b6efc601ccd-multus-daemon-config\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.015251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.013852    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovnkube-config\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.016290 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.016267    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58d8012a-5618-4d1d-aa6f-bfae87401dfc-ovn-node-metrics-cert\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.040582 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.040518    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2pcf\" (UniqueName: \"kubernetes.io/projected/dc93d94a-3701-44a8-8274-ac78bdca6b3f-kube-api-access-h2pcf\") pod \"aws-ebs-csi-driver-node-rt222\" (UID: \"dc93d94a-3701-44a8-8274-ac78bdca6b3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.040582 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.040557    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzkvp\" (UniqueName: \"kubernetes.io/projected/58d8012a-5618-4d1d-aa6f-bfae87401dfc-kube-api-access-bzkvp\") pod \"ovnkube-node-xmv42\" (UID: \"58d8012a-5618-4d1d-aa6f-bfae87401dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.041355 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.041321    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhmzt\" (UniqueName: \"kubernetes.io/projected/e0959543-1c8b-419f-b476-1d92e3c168d0-kube-api-access-zhmzt\") pod \"iptables-alerter-xm8rd\" (UID: \"e0959543-1c8b-419f-b476-1d92e3c168d0\") " pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.042464 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.042440    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jkwc\" (UniqueName: \"kubernetes.io/projected/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-kube-api-access-9jkwc\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:48.078079 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.078052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"61dad3d90ea1a5450d51ca0838b0d656","Type":"ContainerStarted","Data":"d95bc11cef0359c8e98b3bffa670ecc8c119e5b2af03f59e9dd1a8235e80df7c"}
Jul 07 13:27:48.114238 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-etc-kubernetes\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.114238 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-kubernetes\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cnibin\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-modprobe-d\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114333    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-etc-kubernetes\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87706558-76f0-4fbe-88d8-69c886cdf9e0-tmp\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114389    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-kubernetes\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlgtj\" (UniqueName: \"kubernetes.io/projected/87706558-76f0-4fbe-88d8-69c886cdf9e0-kube-api-access-zlgtj\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114407    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cnibin\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8db6f404-a4ed-47b3-9850-eb6b8b776e74-hosts-file\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-host\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.114479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db6f404-a4ed-47b3-9850-eb6b8b776e74-tmp-dir\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114525    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-modprobe-d\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114552    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-host\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114621    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8db6f404-a4ed-47b3-9850-eb6b8b776e74-hosts-file\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-multus-certs\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-system-cni-dir\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f8332404-cb66-4442-a5e4-59b4ea994ab7-kubelet-config\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-run\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114764    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cni-binary-copy\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114808    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db6f404-a4ed-47b3-9850-eb6b8b776e74-tmp-dir\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114857    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-multus-certs\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-var-lib-kubelet\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-serviceca\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114913    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-var-lib-kubelet\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-k8s-cni-cncf-io\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114962    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-cni-multus\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.115064 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-sys\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da30c936-4b65-4167-8891-2b6efc601ccd-cni-binary-copy\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-hostroot\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115066    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-run\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114918    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f8332404-cb66-4442-a5e4-59b4ea994ab7-kubelet-config\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.114808    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-system-cni-dir\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115115    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f8332404-cb66-4442-a5e4-59b4ea994ab7-dbus\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c112cf11-8255-4826-906e-0b8a815f125b-agent-certs\") pod \"konnectivity-agent-48bvw\" (UID: \"c112cf11-8255-4826-906e-0b8a815f125b\") " pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115151    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-k8s-cni-cncf-io\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysconfig\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115223    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysconfig\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115270    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-serviceca\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115271    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-hostroot\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115325    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-cni-multus\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115336    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115362    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-sys\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115391    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxc7j\" (UniqueName: \"kubernetes.io/projected/da30c936-4b65-4167-8891-2b6efc601ccd-kube-api-access-pxc7j\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115410    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.116023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115413    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-cni-binary-copy\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-tuned\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115438    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f8332404-cb66-4442-a5e4-59b4ea994ab7-dbus\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.115517    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.115602    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:48.615579932 +0000 UTC m=+3.148485644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-conf-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115752    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da30c936-4b65-4167-8891-2b6efc601ccd-cni-binary-copy\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115783    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysctl-conf\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115802    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-conf-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-netns\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c112cf11-8255-4826-906e-0b8a815f125b-konnectivity-ca\") pod \"konnectivity-agent-48bvw\" (UID: \"c112cf11-8255-4826-906e-0b8a815f125b\") " pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115839    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysctl-d\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115923    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-run-netns\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115930    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7xkj\" (UniqueName: \"kubernetes.io/projected/8db6f404-a4ed-47b3-9850-eb6b8b776e74-kube-api-access-n7xkj\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115956    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysctl-conf\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.116928 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.115955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-os-release\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116029    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-cni-bin\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/da30c936-4b65-4167-8891-2b6efc601ccd-multus-daemon-config\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116103    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-sysctl-d\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-systemd\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-host\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-os-release\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prvwn\" (UniqueName: \"kubernetes.io/projected/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-kube-api-access-prvwn\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116211    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-os-release\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116240    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-cni-bin\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-cni-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116267    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-systemd\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116283    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-cnibin\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116302    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-cni-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116311    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-socket-dir-parent\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116383    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-cnibin\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116454    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-os-release\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116505    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-multus-socket-dir-parent\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.117824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116543    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-host\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116550    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c112cf11-8255-4826-906e-0b8a815f125b-konnectivity-ca\") pod \"konnectivity-agent-48bvw\" (UID: \"c112cf11-8255-4826-906e-0b8a815f125b\") " pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116573    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-kubelet\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116608    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-lib-modules\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116636    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2x9t\" (UniqueName: \"kubernetes.io/projected/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-kube-api-access-q2x9t\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116658    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-host-var-lib-kubelet\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-system-cni-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116785    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da30c936-4b65-4167-8891-2b6efc601ccd-system-cni-dir\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116789    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/da30c936-4b65-4167-8891-2b6efc601ccd-multus-daemon-config\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.116816    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87706558-76f0-4fbe-88d8-69c886cdf9e0-lib-modules\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.117751    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.118766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.118290    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87706558-76f0-4fbe-88d8-69c886cdf9e0-tmp\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.119379 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.119148    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87706558-76f0-4fbe-88d8-69c886cdf9e0-etc-tuned\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.119379 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.119234    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c112cf11-8255-4826-906e-0b8a815f125b-agent-certs\") pod \"konnectivity-agent-48bvw\" (UID: \"c112cf11-8255-4826-906e-0b8a815f125b\") " pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.147034 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.146954    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:27:48.147034 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.146992    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:27:48.147034 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.147011    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:48.147277 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.146954    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlgtj\" (UniqueName: \"kubernetes.io/projected/87706558-76f0-4fbe-88d8-69c886cdf9e0-kube-api-access-zlgtj\") pod \"tuned-5qvr8\" (UID: \"87706558-76f0-4fbe-88d8-69c886cdf9e0\") " pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.147277 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.147087    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:27:48.647063775 +0000 UTC m=+3.179969501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:48.147277 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.147085    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2x9t\" (UniqueName: \"kubernetes.io/projected/7073646d-a4f6-4a6d-ad97-fe2ad699b39a-kube-api-access-q2x9t\") pod \"node-ca-xhntt\" (UID: \"7073646d-a4f6-4a6d-ad97-fe2ad699b39a\") " pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.149295 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.149269    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7xkj\" (UniqueName: \"kubernetes.io/projected/8db6f404-a4ed-47b3-9850-eb6b8b776e74-kube-api-access-n7xkj\") pod \"node-resolver-5jqwk\" (UID: \"8db6f404-a4ed-47b3-9850-eb6b8b776e74\") " pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.150981 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.150947    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prvwn\" (UniqueName: \"kubernetes.io/projected/7f9a5066-7b52-4696-b4b1-7cb4f0e25598-kube-api-access-prvwn\") pod \"multus-additional-cni-plugins-7r4gz\" (UID: \"7f9a5066-7b52-4696-b4b1-7cb4f0e25598\") " pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.152747 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.152725    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxc7j\" (UniqueName: \"kubernetes.io/projected/da30c936-4b65-4167-8891-2b6efc601ccd-kube-api-access-pxc7j\") pod \"multus-vh9cj\" (UID: \"da30c936-4b65-4167-8891-2b6efc601ccd\") " pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.210896 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.210864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xm8rd"
Jul 07 13:27:48.216680 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.216649    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:27:48.228212 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.228184    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222"
Jul 07 13:27:48.233856 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.233831    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5qvr8"
Jul 07 13:27:48.245460 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.245436    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5jqwk"
Jul 07 13:27:48.252652 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.252616    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:27:48.260644 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.260624    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xhntt"
Jul 07 13:27:48.269594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.269570    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7r4gz"
Jul 07 13:27:48.275627 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.275606    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vh9cj"
Jul 07 13:27:48.520378 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.520301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:48.520532 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.520475    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:48.520586 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.520553    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:49.520533959 +0000 UTC m=+4.053439664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:48.620758 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.620716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:48.620955 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.620887    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:48.621008 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.620966    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:49.620943777 +0000 UTC m=+4.153849512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:48.721500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.721462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:48.721651 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.721637    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:27:48.721725 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.721662    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:27:48.721725 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.721676    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:48.721901 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:48.721743    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:27:49.721723881 +0000 UTC m=+4.254629590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:48.963550 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.963446    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:22:48 +0000 UTC" deadline="2027-12-11 16:40:34.680774933 +0000 UTC"
Jul 07 13:27:48.963550 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:48.963484    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12531h12m45.717296814s"
Jul 07 13:27:49.020298 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.020257    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc112cf11_8255_4826_906e_0b8a815f125b.slice/crio-e78777cbb677e3ccca464a9a6beca0135769955e081d8d7ebcf610747a729972 WatchSource:0}: Error finding container e78777cbb677e3ccca464a9a6beca0135769955e081d8d7ebcf610747a729972: Status 404 returned error can't find the container with id e78777cbb677e3ccca464a9a6beca0135769955e081d8d7ebcf610747a729972
Jul 07 13:27:49.025851 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.025827    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db6f404_a4ed_47b3_9850_eb6b8b776e74.slice/crio-10609c73b5ca4665a2e2cc4525e15c66beba3824c7d5f986cef8daa4cca4d587 WatchSource:0}: Error finding container 10609c73b5ca4665a2e2cc4525e15c66beba3824c7d5f986cef8daa4cca4d587: Status 404 returned error can't find the container with id 10609c73b5ca4665a2e2cc4525e15c66beba3824c7d5f986cef8daa4cca4d587
Jul 07 13:27:49.027583 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.027548    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc93d94a_3701_44a8_8274_ac78bdca6b3f.slice/crio-c7bf92eff1e6462c8ac468cbf095ad1383a382341338b7ddbd0d94344515b6d7 WatchSource:0}: Error finding container c7bf92eff1e6462c8ac468cbf095ad1383a382341338b7ddbd0d94344515b6d7: Status 404 returned error can't find the container with id c7bf92eff1e6462c8ac468cbf095ad1383a382341338b7ddbd0d94344515b6d7
Jul 07 13:27:49.028374 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.028247    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7073646d_a4f6_4a6d_ad97_fe2ad699b39a.slice/crio-d9f4534d38433db09f712f6477677a35fe59997d92694cfcd0c68b11624a4e34 WatchSource:0}: Error finding container d9f4534d38433db09f712f6477677a35fe59997d92694cfcd0c68b11624a4e34: Status 404 returned error can't find the container with id d9f4534d38433db09f712f6477677a35fe59997d92694cfcd0c68b11624a4e34
Jul 07 13:27:49.029534 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.029377    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87706558_76f0_4fbe_88d8_69c886cdf9e0.slice/crio-9b4fff3e13612c59cd79506e21142933dab30ca7e5ad085b3b91200cf462d5a1 WatchSource:0}: Error finding container 9b4fff3e13612c59cd79506e21142933dab30ca7e5ad085b3b91200cf462d5a1: Status 404 returned error can't find the container with id 9b4fff3e13612c59cd79506e21142933dab30ca7e5ad085b3b91200cf462d5a1
Jul 07 13:27:49.032395 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.032358    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0959543_1c8b_419f_b476_1d92e3c168d0.slice/crio-31aa6bc4370957c7d8a88688b5ffab79c70401ffa202c5e862b2aa0043462b16 WatchSource:0}: Error finding container 31aa6bc4370957c7d8a88688b5ffab79c70401ffa202c5e862b2aa0043462b16: Status 404 returned error can't find the container with id 31aa6bc4370957c7d8a88688b5ffab79c70401ffa202c5e862b2aa0043462b16
Jul 07 13:27:49.033840 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.033755    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9a5066_7b52_4696_b4b1_7cb4f0e25598.slice/crio-47ad17133eca308c1d9ba859b3adce641a11920ea1dd8cca1bf2e6ab046207cc WatchSource:0}: Error finding container 47ad17133eca308c1d9ba859b3adce641a11920ea1dd8cca1bf2e6ab046207cc: Status 404 returned error can't find the container with id 47ad17133eca308c1d9ba859b3adce641a11920ea1dd8cca1bf2e6ab046207cc
Jul 07 13:27:49.035212 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.035167    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda30c936_4b65_4167_8891_2b6efc601ccd.slice/crio-2dd7ade2969e93ca6d8389d4e2c6dcd32beae0f7a3c9b7c6c3107a1b2e2868b5 WatchSource:0}: Error finding container 2dd7ade2969e93ca6d8389d4e2c6dcd32beae0f7a3c9b7c6c3107a1b2e2868b5: Status 404 returned error can't find the container with id 2dd7ade2969e93ca6d8389d4e2c6dcd32beae0f7a3c9b7c6c3107a1b2e2868b5
Jul 07 13:27:49.038132 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:27:49.038084    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d8012a_5618_4d1d_aa6f_bfae87401dfc.slice/crio-79a0f296732c19ac633475bec459c043c6749c25c1c5b881e514d69ee1c3674e WatchSource:0}: Error finding container 79a0f296732c19ac633475bec459c043c6749c25c1c5b881e514d69ee1c3674e: Status 404 returned error can't find the container with id 79a0f296732c19ac633475bec459c043c6749c25c1c5b881e514d69ee1c3674e
Jul 07 13:27:49.073048 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.073021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:49.073188 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.073157    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:27:49.081029 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.080993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5qvr8" event={"ID":"87706558-76f0-4fbe-88d8-69c886cdf9e0","Type":"ContainerStarted","Data":"9b4fff3e13612c59cd79506e21142933dab30ca7e5ad085b3b91200cf462d5a1"}
Jul 07 13:27:49.082147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.082122    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222" event={"ID":"dc93d94a-3701-44a8-8274-ac78bdca6b3f","Type":"ContainerStarted","Data":"c7bf92eff1e6462c8ac468cbf095ad1383a382341338b7ddbd0d94344515b6d7"}
Jul 07 13:27:49.083152 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.083129    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vh9cj" event={"ID":"da30c936-4b65-4167-8891-2b6efc601ccd","Type":"ContainerStarted","Data":"2dd7ade2969e93ca6d8389d4e2c6dcd32beae0f7a3c9b7c6c3107a1b2e2868b5"}
Jul 07 13:27:49.084120 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.084099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xm8rd" event={"ID":"e0959543-1c8b-419f-b476-1d92e3c168d0","Type":"ContainerStarted","Data":"31aa6bc4370957c7d8a88688b5ffab79c70401ffa202c5e862b2aa0043462b16"}
Jul 07 13:27:49.085400 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.085302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xhntt" event={"ID":"7073646d-a4f6-4a6d-ad97-fe2ad699b39a","Type":"ContainerStarted","Data":"d9f4534d38433db09f712f6477677a35fe59997d92694cfcd0c68b11624a4e34"}
Jul 07 13:27:49.086375 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.086356    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5jqwk" event={"ID":"8db6f404-a4ed-47b3-9850-eb6b8b776e74","Type":"ContainerStarted","Data":"10609c73b5ca4665a2e2cc4525e15c66beba3824c7d5f986cef8daa4cca4d587"}
Jul 07 13:27:49.087392 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.087372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-48bvw" event={"ID":"c112cf11-8255-4826-906e-0b8a815f125b","Type":"ContainerStarted","Data":"e78777cbb677e3ccca464a9a6beca0135769955e081d8d7ebcf610747a729972"}
Jul 07 13:27:49.088485 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.088461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"79a0f296732c19ac633475bec459c043c6749c25c1c5b881e514d69ee1c3674e"}
Jul 07 13:27:49.091162 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.091130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerStarted","Data":"47ad17133eca308c1d9ba859b3adce641a11920ea1dd8cca1bf2e6ab046207cc"}
Jul 07 13:27:49.531005 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.530619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:49.531005 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.530850    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:49.531232 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.531049    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:51.531027416 +0000 UTC m=+6.063933121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:49.632281 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.632239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:49.632477 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.632459    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:49.632560 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.632535    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:51.63251362 +0000 UTC m=+6.165419332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:49.733076 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.733034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:49.733287 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.733256    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:27:49.733287 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.733279    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:27:49.733416 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.733292    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:49.733416 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:49.733356    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:27:51.733336971 +0000 UTC m=+6.266242676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:49.964229 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.964161    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:22:48 +0000 UTC" deadline="2027-12-09 23:07:00.227113091 +0000 UTC"
Jul 07 13:27:49.964229 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:49.964228    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12489h39m10.262888642s"
Jul 07 13:27:50.074942 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:50.074912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:50.075132 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:50.075049    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:27:50.075464 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:50.075443    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:50.075565 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:50.075547    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:27:50.110199 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:50.110155    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" event={"ID":"1c857f80ddb13de0ede08c410a7a78ba","Type":"ContainerStarted","Data":"9c9db9f187db04a1465a58328b33509759f716a8ccabe17935dc365a562e2db8"}
Jul 07 13:27:50.130398 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:50.129756    2582 generic.go:356] "Generic (PLEG): container finished" podID="61dad3d90ea1a5450d51ca0838b0d656" containerID="4f5ed75946ed30b6a0aa8f89c18d957ea3327c26efe071bc34f8d5d0b868bd43" exitCode=0
Jul 07 13:27:50.130398 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:50.129811    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"61dad3d90ea1a5450d51ca0838b0d656","Type":"ContainerDied","Data":"4f5ed75946ed30b6a0aa8f89c18d957ea3327c26efe071bc34f8d5d0b868bd43"}
Jul 07 13:27:50.172362 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:50.171807    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" podStartSLOduration=3.171782344 podStartE2EDuration="3.171782344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:27:50.133453894 +0000 UTC m=+4.666359625" watchObservedRunningTime="2026-07-07 13:27:50.171782344 +0000 UTC m=+4.704688076"
Jul 07 13:27:51.072824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:51.072318    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:51.072824 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.072461    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:27:51.153172 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:51.153138    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"61dad3d90ea1a5450d51ca0838b0d656","Type":"ContainerStarted","Data":"9ff0621fffb5c0473df601ceb5cefe04bbdecf826ac0cd4a9648445c2f90da65"}
Jul 07 13:27:51.170193 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:51.169846    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" podStartSLOduration=4.169825012 podStartE2EDuration="4.169825012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:27:51.169458309 +0000 UTC m=+5.702364059" watchObservedRunningTime="2026-07-07 13:27:51.169825012 +0000 UTC m=+5.702730738"
Jul 07 13:27:51.551166 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:51.550468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:51.551166 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.550667    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:51.551166 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.550728    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:55.550710418 +0000 UTC m=+10.083616124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:51.652289 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:51.651739    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:51.652289 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.651895    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:51.652289 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.651964    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:27:55.651946232 +0000 UTC m=+10.184851947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:51.752312 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:51.752274    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:51.752617 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.752595    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:27:51.752693 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.752628    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:27:51.752693 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.752641    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:51.752831 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:51.752701    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:27:55.752683451 +0000 UTC m=+10.285589171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:52.073101 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:52.073065    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:52.073554 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:52.073076    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:52.073554 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:52.073198    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:27:52.073554 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:52.073277    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:27:53.073073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:53.073043    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:53.073270 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:53.073186    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:27:54.073233 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:54.072907    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:54.073233 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:54.073053    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:27:54.073233 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:54.073098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:54.073233 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:54.073169    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:27:55.072555 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:55.072518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:55.072727 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.072664    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:27:55.583751 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:55.583714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:55.584220 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.583880    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:55.584220 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.583942    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:03.583927645 +0000 UTC m=+18.116833348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:27:55.684961 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:55.684921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:55.685127 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.685064    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:55.685184 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.685149    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:03.68512844 +0000 UTC m=+18.218034158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:27:55.786741 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:55.786394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:55.786741 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.786566    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:27:55.786741 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.786585    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:27:55.786741 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.786599    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:55.786741 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:55.786661    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:28:03.786640802 +0000 UTC m=+18.319546511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:27:56.073925 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:56.073273    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:56.073925 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:56.073394    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:27:56.073925 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:56.073446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:56.073925 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:56.073594    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:27:57.072961 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:57.072923    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:57.073417 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:57.073077    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:27:58.073725 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:58.073506    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:27:58.073725 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:58.073688    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:27:58.074265 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:58.073735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:27:58.074265 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:58.073859    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:27:59.072883 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:27:59.072846    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:27:59.073049 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:27:59.072962    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:00.073202 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:00.073003    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:00.073202 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:00.073012    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:00.073202 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:00.073132    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:00.073202 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:00.073195    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:01.072939 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:01.072904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:01.073128 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:01.073036    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:02.072911 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:02.072851    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:02.073338 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:02.072974    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:02.073338 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:02.073051    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:02.073338 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:02.073136    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:03.072511 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:03.072472    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:03.072731 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.072627    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:03.649823 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:03.649792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:03.650216 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.649880    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:28:03.650216 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.649934    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:19.649918543 +0000 UTC m=+34.182824259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:28:03.750094 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:03.750058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:03.750211 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.750175    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:28:03.750263 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.750237    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:19.750223256 +0000 UTC m=+34.283128979 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:28:03.850411 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:03.850383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:03.850576 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.850530    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:28:03.850576 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.850547    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:28:03.850576 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.850556    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:28:03.850695 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:03.850606    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:28:19.850589187 +0000 UTC m=+34.383494890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:28:04.072519 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:04.072438    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:04.072679 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:04.072440    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:04.072679 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:04.072564    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:04.072679 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:04.072653    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:05.072923 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:05.072894    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:05.073365 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:05.073004    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:06.073985 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:06.073945    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:06.074453 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:06.074056    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:06.074453 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:06.074110    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:06.074453 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:06.074234    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:07.072965 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:07.072928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:07.073148 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:07.073060    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:08.072948 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.072442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:08.072948 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:08.072733    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:08.072948 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.072623    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:08.072948 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:08.072892    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:08.189481 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.189445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vh9cj" event={"ID":"da30c936-4b65-4167-8891-2b6efc601ccd","Type":"ContainerStarted","Data":"a6a57f111f11c6c0cf02ca869bff28183a9e5b6f630906cf968aedcd01f1332f"}
Jul 07 13:28:08.190976 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.190926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xhntt" event={"ID":"7073646d-a4f6-4a6d-ad97-fe2ad699b39a","Type":"ContainerStarted","Data":"45c16d5258a1e9fc0bf5fe55aa3dd48c8f42373fb37703be230ff2f4f4046512"}
Jul 07 13:28:08.192544 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.192520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5jqwk" event={"ID":"8db6f404-a4ed-47b3-9850-eb6b8b776e74","Type":"ContainerStarted","Data":"330188bbd07ca974362a52ef8dc1fde98f18b7ddab77ecb995b984ab6dd364e8"}
Jul 07 13:28:08.194397 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.193999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-48bvw" event={"ID":"c112cf11-8255-4826-906e-0b8a815f125b","Type":"ContainerStarted","Data":"5d7c78ba5989b6e9d9de0298b97297ad3b4036daf27cfe42d8593805518bda9e"}
Jul 07 13:28:08.195791 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.195744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"8db3632bb429b1f4a6df99110e4c68e273434d2255378c0567499a2518ff743a"}
Jul 07 13:28:08.195900 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.195826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"4dcad5eb6c6a6b89f5c00aad3aa29ff16299888fb5a4ecd778cd918806693ab1"}
Jul 07 13:28:08.197216 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.197194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerStarted","Data":"d4679c3950db3791dcd41680610b31eb8aa0b5c0a4acd4a128b6b341b007e76d"}
Jul 07 13:28:08.199096 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.199019    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5qvr8" event={"ID":"87706558-76f0-4fbe-88d8-69c886cdf9e0","Type":"ContainerStarted","Data":"9d735dd0f3c0e19b756fcc422fecc651b413b0b9d3bd8cb91ef951f6e8e991eb"}
Jul 07 13:28:08.200499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.200478    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222" event={"ID":"dc93d94a-3701-44a8-8274-ac78bdca6b3f","Type":"ContainerStarted","Data":"0287e66570ce14936e7f835e9ccf350031ce3a3a650b599e478ca9b01cdd46a1"}
Jul 07 13:28:08.231230 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.231058    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vh9cj" podStartSLOduration=3.308243342 podStartE2EDuration="22.231042708s" totalImagesPullingTime="18.922799366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:08.212496486 +0000 UTC m=+22.745402212" watchObservedRunningTime="2026-07-07 13:28:08.231042708 +0000 UTC m=+22.763948509"
Jul 07 13:28:08.250431 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.250379    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5qvr8" podStartSLOduration=3.381168624 podStartE2EDuration="22.250361575s" totalImagesPullingTime="18.869192951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:08.230892843 +0000 UTC m=+22.763798568" watchObservedRunningTime="2026-07-07 13:28:08.250361575 +0000 UTC m=+22.783267299"
Jul 07 13:28:08.250603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.250581    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5jqwk" podStartSLOduration=3.407943934 podStartE2EDuration="22.250572361s" totalImagesPullingTime="18.842628427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:08.250353183 +0000 UTC m=+22.783258909" watchObservedRunningTime="2026-07-07 13:28:08.250572361 +0000 UTC m=+22.783478087"
Jul 07 13:28:08.274828 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.274768    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xhntt" podStartSLOduration=3.434642713 podStartE2EDuration="22.274752977s" totalImagesPullingTime="18.840110264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:08.270811308 +0000 UTC m=+22.803717034" watchObservedRunningTime="2026-07-07 13:28:08.274752977 +0000 UTC m=+22.807658701"
Jul 07 13:28:08.290485 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:08.290436    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-48bvw" podStartSLOduration=7.603263191 podStartE2EDuration="22.290417889s" totalImagesPullingTime="14.687154698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:08.289844506 +0000 UTC m=+22.822750252" watchObservedRunningTime="2026-07-07 13:28:08.290417889 +0000 UTC m=+22.823323616"
Jul 07 13:28:09.073010 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.072980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:09.073730 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:09.073100    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:09.205091 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.205052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xm8rd" event={"ID":"e0959543-1c8b-419f-b476-1d92e3c168d0","Type":"ContainerStarted","Data":"4d8dfacd0eb4debde2ab3fe76054b0d88857d889b116f14cbdb288b1c26de0ce"}
Jul 07 13:28:09.208490 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.208454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"d6ceb5a0db9dd29390efca75b63c9dd99a932d13adc199f4cfcb209862381702"}
Jul 07 13:28:09.208636 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.208540    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"f0b94567dbc88c9423ed7b848057010faa3d5b3916f3af1fafc73c674f5ff56a"}
Jul 07 13:28:09.208636 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.208592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"68fea7b1a3ce5eec3789997e4ceb809df9bdf7069d65095a0de828a44f3c8cd1"}
Jul 07 13:28:09.208756 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.208639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"3d9921067f951c16eb9a084a299d005789edf9db7c310b0e46a538d8507bc70a"}
Jul 07 13:28:09.210118 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.210092    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f9a5066-7b52-4696-b4b1-7cb4f0e25598" containerID="d4679c3950db3791dcd41680610b31eb8aa0b5c0a4acd4a128b6b341b007e76d" exitCode=0
Jul 07 13:28:09.210234 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.210171    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerDied","Data":"d4679c3950db3791dcd41680610b31eb8aa0b5c0a4acd4a128b6b341b007e76d"}
Jul 07 13:28:09.220692 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.220641    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xm8rd" podStartSLOduration=4.35471498 podStartE2EDuration="23.220620847s" totalImagesPullingTime="18.865905867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:09.220592313 +0000 UTC m=+23.753498050" watchObservedRunningTime="2026-07-07 13:28:09.220620847 +0000 UTC m=+23.753526573"
Jul 07 13:28:09.315173 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.315143    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:28:09.315836 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.315812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:28:09.450688 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.450663    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 13:28:09.965214 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.965091    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T13:28:09.450684641Z","UUID":"4bc98d4c-68ea-4469-92d3-91c3c498ba73","Handler":null,"Name":"","Endpoint":""}
Jul 07 13:28:09.966615 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.966573    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 13:28:09.966764 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:09.966629    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 13:28:10.073467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:10.073428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:10.073996 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:10.073468    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:10.073996 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:10.073541    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:10.073996 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:10.073676    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:10.214254 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:10.214215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222" event={"ID":"dc93d94a-3701-44a8-8274-ac78bdca6b3f","Type":"ContainerStarted","Data":"9152070bde87e64a8d537ed827f4630c7803a487fcd136a4b0f0846a8e8d68e8"}
Jul 07 13:28:11.073010 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:11.072980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:11.073153 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:11.073098    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:11.220622 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:11.220532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"628fc1276b47d8d90698281ba05e033cc2211ef1c2ebebffb6cff56c2495132e"}
Jul 07 13:28:11.222832 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:11.222803    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222" event={"ID":"dc93d94a-3701-44a8-8274-ac78bdca6b3f","Type":"ContainerStarted","Data":"5461750ab81c3bacbf6b56fdcd0c2a75f1422ede203dab11fcdb7241a5263193"}
Jul 07 13:28:11.222976 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:11.222839    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 13:28:11.244340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:11.244283    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rt222" podStartSLOduration=3.856383149 podStartE2EDuration="25.244264994s" totalImagesPullingTime="21.387881845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:11.243644212 +0000 UTC m=+25.776549951" watchObservedRunningTime="2026-07-07 13:28:11.244264994 +0000 UTC m=+25.777170726"
Jul 07 13:28:12.072817 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:12.072766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:12.072990 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:12.072766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:12.072990 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:12.072928    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:12.073089 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:12.072988    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:13.073493 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:13.073246    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:13.073903 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:13.073611    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:14.073023 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.072950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:14.073155 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.072950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:14.073155 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:14.073082    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:14.073155 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:14.073120    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:14.095153 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.095126    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:28:14.095831 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.095228    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 13:28:14.095831 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.095655    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-48bvw"
Jul 07 13:28:14.232401 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.232367    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" event={"ID":"58d8012a-5618-4d1d-aa6f-bfae87401dfc","Type":"ContainerStarted","Data":"b83506edd0427ac3751c8c8687d8cb34137d2a6343e7de50e9294281b993c0a8"}
Jul 07 13:28:14.232605 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.232582    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:28:14.232706 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.232656    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:28:14.234140 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.234116    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f9a5066-7b52-4696-b4b1-7cb4f0e25598" containerID="e725bb27716340199b32b4b4c5527d439796dbdb23b86d24bcae7c4940fdc9e9" exitCode=0
Jul 07 13:28:14.234280 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.234211    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerDied","Data":"e725bb27716340199b32b4b4c5527d439796dbdb23b86d24bcae7c4940fdc9e9"}
Jul 07 13:28:14.248815 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.248792    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:28:14.321163 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:14.321118    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42" podStartSLOduration=9.38253899 podStartE2EDuration="28.321102634s" totalImagesPullingTime="18.938563644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:14.278035174 +0000 UTC m=+28.810940898" watchObservedRunningTime="2026-07-07 13:28:14.321102634 +0000 UTC m=+28.854008360"
Jul 07 13:28:15.073451 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.073268    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:15.073676 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:15.073534    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:15.236336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.236307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:28:15.250325 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.250295    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:28:15.747555 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.747477    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ttjh4"]
Jul 07 13:28:15.747722 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.747603    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:15.747722 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:15.747705    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:15.748260 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.748227    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pjn5s"]
Jul 07 13:28:15.748400 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.748330    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:15.748463 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:15.748419    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:15.748864 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.748843    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sg4vv"]
Jul 07 13:28:15.748955 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:15.748925    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:15.749015 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:15.748998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:16.242751 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:16.242663    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f9a5066-7b52-4696-b4b1-7cb4f0e25598" containerID="7a8b8bcc8bb0a0a59435b88e3c83b0a518b7ce9357af6af18ddaae7bcb00bc3e" exitCode=0
Jul 07 13:28:16.242751 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:16.242703    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerDied","Data":"7a8b8bcc8bb0a0a59435b88e3c83b0a518b7ce9357af6af18ddaae7bcb00bc3e"}
Jul 07 13:28:17.072905 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:17.072879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:17.073038 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:17.072973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:17.073038 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:17.072981    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:17.073038 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:17.073002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:17.073191 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:17.073088    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:17.073191 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:17.073157    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:18.012918 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:18.012732    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:18.123640 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:18.123606    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:18.227016 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:18.226983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:18.248577 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:18.248548    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f9a5066-7b52-4696-b4b1-7cb4f0e25598" containerID="698a566e9ba844c4326aaf846807264e7ff5c695bd8a1b851ef9ae3a320dc811" exitCode=0
Jul 07 13:28:18.248722 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:18.248606    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerDied","Data":"698a566e9ba844c4326aaf846807264e7ff5c695bd8a1b851ef9ae3a320dc811"}
Jul 07 13:28:18.296705 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:18.296633    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:19.004641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.004530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:19.072650 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.072613    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:19.073103 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.072736    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:19.073103 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.072749    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttjh4" podUID="580dbd22-63bc-42c6-b087-f118b618e48d"
Jul 07 13:28:19.073103 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.072768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:19.073103 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.072881    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjn5s" podUID="f8332404-cb66-4442-a5e4-59b4ea994ab7"
Jul 07 13:28:19.073103 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.072951    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg4vv" podUID="4d32c03b-8bd2-4e94-83c6-d6c6a5927d47"
Jul 07 13:28:19.103535 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.103504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:19.669105 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.669068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:19.669293 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.669216    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:28:19.669293 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.669281    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs podName:4d32c03b-8bd2-4e94-83c6-d6c6a5927d47 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:51.669262725 +0000 UTC m=+66.202168430 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs") pod "network-metrics-daemon-sg4vv" (UID: "4d32c03b-8bd2-4e94-83c6-d6c6a5927d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:28:19.769583 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.769546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:19.769734 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.769654    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:28:19.769734 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.769716    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret podName:f8332404-cb66-4442-a5e4-59b4ea994ab7 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:51.76970034 +0000 UTC m=+66.302606043 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret") pod "global-pull-secret-syncer-pjn5s" (UID: "f8332404-cb66-4442-a5e4-59b4ea994ab7") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:28:19.870259 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:19.870216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:19.870429 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.870341    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:28:19.870429 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.870358    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:28:19.870429 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.870367    2582 projected.go:196] Error preparing data for projected volume kube-api-access-v4bjh for pod openshift-network-diagnostics/network-check-target-ttjh4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:28:19.870429 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:19.870414    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh podName:580dbd22-63bc-42c6-b087-f118b618e48d nodeName:}" failed. No retries permitted until 2026-07-07 13:28:51.870401743 +0000 UTC m=+66.403307445 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v4bjh" (UniqueName: "kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh") pod "network-check-target-ttjh4" (UID: "580dbd22-63bc-42c6-b087-f118b618e48d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:28:20.712987 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.712921    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeReady"
Jul 07 13:28:20.713366 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.713049    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 13:28:20.753670 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.753641    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5486f79654-d4kzb"]
Jul 07 13:28:20.755570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.755548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.759266 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.759243    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mmkl7\""
Jul 07 13:28:20.759482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.759467    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 13:28:20.759558 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.759507    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 13:28:20.763552 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.763533    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 13:28:20.765435 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.765415    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7lgc2"]
Jul 07 13:28:20.768698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.768093    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.769213 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.769193    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 13:28:20.770991 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.770970    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 13:28:20.771252 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.771233    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rgbgq\""
Jul 07 13:28:20.771484 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.771469    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 13:28:20.775800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.775763    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j8n8j"]
Jul 07 13:28:20.777838 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.777821    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.781209 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.781188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5486f79654-d4kzb"]
Jul 07 13:28:20.781618 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.781497    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 13:28:20.781618 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.781540    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 13:28:20.781618 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.781615    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 13:28:20.781870 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.781835    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-twdpw\""
Jul 07 13:28:20.790053 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.790031    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lgc2"]
Jul 07 13:28:20.800612 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.800567    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j8n8j"]
Jul 07 13:28:20.878090 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878059    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-installation-pull-secrets\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878090 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-bound-sa-token\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.878336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lpfz\" (UniqueName: \"kubernetes.io/projected/b3f1c729-7dac-4fc7-bd63-780b499660b9-kube-api-access-6lpfz\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.878336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878248    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhwzv\" (UniqueName: \"kubernetes.io/projected/76b41427-1a41-4266-b6ef-6948526ec2be-kube-api-access-xhwzv\") pod \"ingress-canary-j8n8j\" (UID: \"76b41427-1a41-4266-b6ef-6948526ec2be\") " pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.878336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-certificates\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878316    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb8k7\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-kube-api-access-nb8k7\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76b41427-1a41-4266-b6ef-6948526ec2be-cert\") pod \"ingress-canary-j8n8j\" (UID: \"76b41427-1a41-4266-b6ef-6948526ec2be\") " pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-trusted-ca\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1c729-7dac-4fc7-bd63-780b499660b9-tmp-dir\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878433    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ad9a0f71-cf78-4021-90c7-f0d118473d98-ca-trust-extracted\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878457    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3f1c729-7dac-4fc7-bd63-780b499660b9-config-volume\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878481    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.878592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.878511    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-image-registry-private-configuration\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.978920 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.978843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.978920 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.978876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lpfz\" (UniqueName: \"kubernetes.io/projected/b3f1c729-7dac-4fc7-bd63-780b499660b9-kube-api-access-6lpfz\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.978920 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.978915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhwzv\" (UniqueName: \"kubernetes.io/projected/76b41427-1a41-4266-b6ef-6948526ec2be-kube-api-access-xhwzv\") pod \"ingress-canary-j8n8j\" (UID: \"76b41427-1a41-4266-b6ef-6948526ec2be\") " pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.979177 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.978937    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-certificates\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979177 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:20.979006    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:28:20.979177 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:20.979069    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls podName:b3f1c729-7dac-4fc7-bd63-780b499660b9 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:21.479049459 +0000 UTC m=+36.011955162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls") pod "dns-default-7lgc2" (UID: "b3f1c729-7dac-4fc7-bd63-780b499660b9") : secret "dns-default-metrics-tls" not found
Jul 07 13:28:20.979177 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb8k7\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-kube-api-access-nb8k7\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979177 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979161    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76b41427-1a41-4266-b6ef-6948526ec2be-cert\") pod \"ingress-canary-j8n8j\" (UID: \"76b41427-1a41-4266-b6ef-6948526ec2be\") " pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-trusted-ca\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1c729-7dac-4fc7-bd63-780b499660b9-tmp-dir\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ad9a0f71-cf78-4021-90c7-f0d118473d98-ca-trust-extracted\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3f1c729-7dac-4fc7-bd63-780b499660b9-config-volume\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979304    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-image-registry-private-configuration\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-installation-pull-secrets\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-bound-sa-token\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979758 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979484    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-certificates\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.979758 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979670    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1c729-7dac-4fc7-bd63-780b499660b9-tmp-dir\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.979935 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:20.979914    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:28:20.979987 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:20.979938    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5486f79654-d4kzb: secret "image-registry-tls" not found
Jul 07 13:28:20.979987 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.979949    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3f1c729-7dac-4fc7-bd63-780b499660b9-config-volume\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.980088 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:20.979993    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls podName:ad9a0f71-cf78-4021-90c7-f0d118473d98 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:21.479975244 +0000 UTC m=+36.012880961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls") pod "image-registry-5486f79654-d4kzb" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98") : secret "image-registry-tls" not found
Jul 07 13:28:20.980230 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.980206    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ad9a0f71-cf78-4021-90c7-f0d118473d98-ca-trust-extracted\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.980645 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.980514    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-trusted-ca\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.983804 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.983761    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-installation-pull-secrets\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.983915 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.983762    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76b41427-1a41-4266-b6ef-6948526ec2be-cert\") pod \"ingress-canary-j8n8j\" (UID: \"76b41427-1a41-4266-b6ef-6948526ec2be\") " pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.983915 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.983762    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-image-registry-private-configuration\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.988814 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.988753    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-bound-sa-token\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:20.989479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.989456    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhwzv\" (UniqueName: \"kubernetes.io/projected/76b41427-1a41-4266-b6ef-6948526ec2be-kube-api-access-xhwzv\") pod \"ingress-canary-j8n8j\" (UID: \"76b41427-1a41-4266-b6ef-6948526ec2be\") " pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:20.989667 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.989649    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lpfz\" (UniqueName: \"kubernetes.io/projected/b3f1c729-7dac-4fc7-bd63-780b499660b9-kube-api-access-6lpfz\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:20.991981 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:20.991963    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb8k7\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-kube-api-access-nb8k7\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:21.073317 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.073280    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:21.073466 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.073412    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:21.073533 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.073284    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:21.079180 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.079155    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4z824\""
Jul 07 13:28:21.079328 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.079310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 13:28:21.079451 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.079427    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 13:28:21.079920 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.079855    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 13:28:21.080033 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.080008    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 13:28:21.080211 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.080157    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k6stv\""
Jul 07 13:28:21.087737 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.087714    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j8n8j"
Jul 07 13:28:21.240676 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.240555    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j8n8j"]
Jul 07 13:28:21.483243 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.483205    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:21.483415 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:21.483360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:21.483415 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:21.483382    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:28:21.483415 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:21.483403    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5486f79654-d4kzb: secret "image-registry-tls" not found
Jul 07 13:28:21.483584 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:21.483484    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls podName:ad9a0f71-cf78-4021-90c7-f0d118473d98 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:22.48346276 +0000 UTC m=+37.016368462 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls") pod "image-registry-5486f79654-d4kzb" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98") : secret "image-registry-tls" not found
Jul 07 13:28:21.483584 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:21.483507    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:28:21.483584 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:21.483561    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls podName:b3f1c729-7dac-4fc7-bd63-780b499660b9 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:22.483546388 +0000 UTC m=+37.016452091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls") pod "dns-default-7lgc2" (UID: "b3f1c729-7dac-4fc7-bd63-780b499660b9") : secret "dns-default-metrics-tls" not found
Jul 07 13:28:22.491465 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:22.491430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:22.492182 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:22.491491    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:22.492182 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:22.491592    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:28:22.492182 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:22.491600    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:28:22.492182 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:22.491625    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5486f79654-d4kzb: secret "image-registry-tls" not found
Jul 07 13:28:22.492182 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:22.491659    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls podName:b3f1c729-7dac-4fc7-bd63-780b499660b9 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:24.491641572 +0000 UTC m=+39.024547277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls") pod "dns-default-7lgc2" (UID: "b3f1c729-7dac-4fc7-bd63-780b499660b9") : secret "dns-default-metrics-tls" not found
Jul 07 13:28:22.492182 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:22.491679    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls podName:ad9a0f71-cf78-4021-90c7-f0d118473d98 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:24.491670703 +0000 UTC m=+39.024576409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls") pod "image-registry-5486f79654-d4kzb" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98") : secret "image-registry-tls" not found
Jul 07 13:28:22.711621 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:22.711451    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b41427_1a41_4266_b6ef_6948526ec2be.slice/crio-8eaa5db09abf562777375baae6a3357c14531483dc75215838dcebe035229a64 WatchSource:0}: Error finding container 8eaa5db09abf562777375baae6a3357c14531483dc75215838dcebe035229a64: Status 404 returned error can't find the container with id 8eaa5db09abf562777375baae6a3357c14531483dc75215838dcebe035229a64
Jul 07 13:28:23.264667 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:23.264630    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f9a5066-7b52-4696-b4b1-7cb4f0e25598" containerID="d5785ba2e3692b78b2f8be788325f6c199f7184f4d7ee0cb554b7383b6d3e313" exitCode=0
Jul 07 13:28:23.264889 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:23.264711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerDied","Data":"d5785ba2e3692b78b2f8be788325f6c199f7184f4d7ee0cb554b7383b6d3e313"}
Jul 07 13:28:23.266071 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:23.266051    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j8n8j" event={"ID":"76b41427-1a41-4266-b6ef-6948526ec2be","Type":"ContainerStarted","Data":"8eaa5db09abf562777375baae6a3357c14531483dc75215838dcebe035229a64"}
Jul 07 13:28:24.271714 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:24.271681    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f9a5066-7b52-4696-b4b1-7cb4f0e25598" containerID="03885b4ba7ea0e280a99a260b77b58b834d0b732897535d688430b1813bdfb7a" exitCode=0
Jul 07 13:28:24.272131 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:24.271739    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerDied","Data":"03885b4ba7ea0e280a99a260b77b58b834d0b732897535d688430b1813bdfb7a"}
Jul 07 13:28:24.508899 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:24.508853    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:24.509045 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:24.508950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:24.509087 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:24.509039    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:28:24.509087 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:24.509060    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:28:24.509087 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:24.509075    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5486f79654-d4kzb: secret "image-registry-tls" not found
Jul 07 13:28:24.509215 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:24.509120    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls podName:b3f1c729-7dac-4fc7-bd63-780b499660b9 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:28.509095829 +0000 UTC m=+43.042001536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls") pod "dns-default-7lgc2" (UID: "b3f1c729-7dac-4fc7-bd63-780b499660b9") : secret "dns-default-metrics-tls" not found
Jul 07 13:28:24.509215 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:24.509139    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls podName:ad9a0f71-cf78-4021-90c7-f0d118473d98 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:28.509129914 +0000 UTC m=+43.042035623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls") pod "image-registry-5486f79654-d4kzb" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98") : secret "image-registry-tls" not found
Jul 07 13:28:25.277326 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:25.277292    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" event={"ID":"7f9a5066-7b52-4696-b4b1-7cb4f0e25598","Type":"ContainerStarted","Data":"777f43fe2dd84c5ae12fad04f6c077c8f7ffb550bec6f813059ee499b68683e3"}
Jul 07 13:28:25.302065 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:25.302018    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7r4gz" podStartSLOduration=8.354637351 podStartE2EDuration="39.301997421s" totalImagesPullingTime="30.94736007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:25.301013762 +0000 UTC m=+39.833919488" watchObservedRunningTime="2026-07-07 13:28:25.301997421 +0000 UTC m=+39.834903129"
Jul 07 13:28:25.426930 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:25.426891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:25.528438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:25.528350    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:25.628515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:25.628478    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:25.969706 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:25.969673    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hl4gp"]
Jul 07 13:28:26.000850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.000825    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hl4gp"]
Jul 07 13:28:26.001015 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.000938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.006689 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.006664    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 13:28:26.007833 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.007811    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 13:28:26.008040 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.008021    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t5s5h\""
Jul 07 13:28:26.008869 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.008842    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 13:28:26.009105 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.009090    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 13:28:26.024297 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.024277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:26.123249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.123216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-data-volume\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.123422 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.123264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-crio-socket\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.123422 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.123283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4wbn\" (UniqueName: \"kubernetes.io/projected/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-api-access-b4wbn\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.123422 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.123381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.123533 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.123433    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.138618 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.138586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:26.224262 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.224262 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-data-volume\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.224470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-crio-socket\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.224470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wbn\" (UniqueName: \"kubernetes.io/projected/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-api-access-b4wbn\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.224470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.224470 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.224318    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:28:26.224669 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224495    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-crio-socket\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.224669 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.224530    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:26.224669 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.224548    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:26.724529879 +0000 UTC m=+41.257435582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : configmap "kube-rbac-proxy" not found
Jul 07 13:28:26.224669 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.224591    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:26.724574262 +0000 UTC m=+41.257479969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:26.224669 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.224616    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-data-volume\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.226636 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.226609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:26.239350 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.239331    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4wbn\" (UniqueName: \"kubernetes.io/projected/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-api-access-b4wbn\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.280500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.280467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j8n8j" event={"ID":"76b41427-1a41-4266-b6ef-6948526ec2be","Type":"ContainerStarted","Data":"d0e02a4c0dfe19e7b3f3e709964923b1ab1a678987d61fe1f80f2b5e12d8c2db"}
Jul 07 13:28:26.302557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.302508    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j8n8j" podStartSLOduration=3.708364347 podStartE2EDuration="6.3024939s" totalImagesPullingTime="2.594129553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:26.302266334 +0000 UTC m=+40.835172059" watchObservedRunningTime="2026-07-07 13:28:26.3024939 +0000 UTC m=+40.835399623"
Jul 07 13:28:26.728240 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.728209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.728392 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:26.728256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:26.728392 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.728380    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:26.728479 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.728431    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:28:26.728479 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.728461    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:27.728443462 +0000 UTC m=+42.261349184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:26.728479 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:26.728478    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:27.728467162 +0000 UTC m=+42.261372865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : configmap "kube-rbac-proxy" not found
Jul 07 13:28:27.129182 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.129149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:27.206983 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.206950    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:27.274328 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.274299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:27.611920 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.611884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:27.690438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.690405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:27.735638 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.735605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:27.735833 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.735657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:27.735833 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:27.735740    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:27.735833 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:27.735741    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:28:27.735990 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:27.735839    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:29.735821606 +0000 UTC m=+44.268727325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:27.735990 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:27.735874    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:29.735858675 +0000 UTC m=+44.268764378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : configmap "kube-rbac-proxy" not found
Jul 07 13:28:27.753805 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.753762    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5jqwk_8db6f404-a4ed-47b3-9850-eb6b8b776e74/dns-node-resolver/0.log"
Jul 07 13:28:27.805846 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.805815    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:27.911814 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:27.911721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:28.007711 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:28.007675    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:28.074824 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:28.074791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:28.141528 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:28.141495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:28.541579 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:28.541543    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:28.541795 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:28.541615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:28.541795 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:28.541702    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:28:28.541795 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:28.541787    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls podName:b3f1c729-7dac-4fc7-bd63-780b499660b9 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:36.541753196 +0000 UTC m=+51.074658898 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls") pod "dns-default-7lgc2" (UID: "b3f1c729-7dac-4fc7-bd63-780b499660b9") : secret "dns-default-metrics-tls" not found
Jul 07 13:28:28.541958 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:28.541707    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:28:28.541958 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:28.541820    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5486f79654-d4kzb: secret "image-registry-tls" not found
Jul 07 13:28:28.541958 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:28.541877    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls podName:ad9a0f71-cf78-4021-90c7-f0d118473d98 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:36.541861452 +0000 UTC m=+51.074767157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls") pod "image-registry-5486f79654-d4kzb" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98") : secret "image-registry-tls" not found
Jul 07 13:28:28.988940 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:28.988904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:29.092633 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.092595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:29.152006 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.151979    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhntt_7073646d-a4f6-4a6d-ad97-fe2ad699b39a/node-ca/0.log"
Jul 07 13:28:29.383035 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.382997    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:29.484714 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.484682    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-ttknt"]
Jul 07 13:28:29.484879 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.484756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:29.513527 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.513505    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-ttknt"]
Jul 07 13:28:29.513639 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.513586    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.517363 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.517345    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 13:28:29.518419 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.518399    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 13:28:29.518509 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.518493    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 13:28:29.518563 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.518505    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6r8m8\""
Jul 07 13:28:29.519036 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.519017    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 13:28:29.549876 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.549842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-484ns\" (UniqueName: \"kubernetes.io/projected/3f875ce8-40e8-457a-800c-b7f22fd2cc70-kube-api-access-484ns\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.550035 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.549883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3f875ce8-40e8-457a-800c-b7f22fd2cc70-signing-key\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.550035 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.549976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3f875ce8-40e8-457a-800c-b7f22fd2cc70-signing-cabundle\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.555178 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.555144    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:29.650420 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.650344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3f875ce8-40e8-457a-800c-b7f22fd2cc70-signing-cabundle\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.650557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.650464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-484ns\" (UniqueName: \"kubernetes.io/projected/3f875ce8-40e8-457a-800c-b7f22fd2cc70-kube-api-access-484ns\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.650557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.650496    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3f875ce8-40e8-457a-800c-b7f22fd2cc70-signing-key\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.651031 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.651012    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3f875ce8-40e8-457a-800c-b7f22fd2cc70-signing-cabundle\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.652818 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.652797    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3f875ce8-40e8-457a-800c-b7f22fd2cc70-signing-key\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.667382 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.667364    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-484ns\" (UniqueName: \"kubernetes.io/projected/3f875ce8-40e8-457a-800c-b7f22fd2cc70-kube-api-access-484ns\") pod \"service-ca-758f599c5-ttknt\" (UID: \"3f875ce8-40e8-457a-800c-b7f22fd2cc70\") " pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.694838 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.694814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:29.750973 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.750946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:29.751100 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.750989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:29.751135 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:29.751092    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:29.751165 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:29.751154    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:33.751140122 +0000 UTC m=+48.284045825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:29.751432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.751416    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:29.822253 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.822216    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-ttknt"
Jul 07 13:28:29.954482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.954456    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j8n8j_76b41427-1a41-4266-b6ef-6948526ec2be/serve-healthcheck-canary/0.log"
Jul 07 13:28:29.967946 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:29.967918    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-ttknt"]
Jul 07 13:28:29.971467 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:29.971440    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f875ce8_40e8_457a_800c_b7f22fd2cc70.slice/crio-77a0d61cfff3bad99442b27043dce729f5457b0cd574f056a80387f2bf4d7d26 WatchSource:0}: Error finding container 77a0d61cfff3bad99442b27043dce729f5457b0cd574f056a80387f2bf4d7d26: Status 404 returned error can't find the container with id 77a0d61cfff3bad99442b27043dce729f5457b0cd574f056a80387f2bf4d7d26
Jul 07 13:28:30.290194 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:30.290099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-ttknt" event={"ID":"3f875ce8-40e8-457a-800c-b7f22fd2cc70","Type":"ContainerStarted","Data":"77a0d61cfff3bad99442b27043dce729f5457b0cd574f056a80387f2bf4d7d26"}
Jul 07 13:28:30.940203 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:30.940169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:31.749747 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:31.749707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:31.850052 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:31.850011    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:33.025068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:33.025036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:33.107537 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:33.107499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:33.568672 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:33.568592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:33.772246 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:33.772214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:33.772396 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:33.772352    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:33.772432 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:33.772409    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls podName:d61a68c1-2c75-40a8-9e32-0e903fbd9d13 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:41.772393945 +0000 UTC m=+56.305299651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hl4gp" (UID: "d61a68c1-2c75-40a8-9e32-0e903fbd9d13") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:28:34.301125 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:34.301093    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-ttknt" event={"ID":"3f875ce8-40e8-457a-800c-b7f22fd2cc70","Type":"ContainerStarted","Data":"70922be232681fd4bf0fc74570394f353b892262763e44ca6fdf6b7a1f4f7657"}
Jul 07 13:28:34.320040 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:34.319987    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-ttknt" podStartSLOduration=2.058118697 podStartE2EDuration="5.319967616s" totalImagesPullingTime="3.261848919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:34.31943266 +0000 UTC m=+48.852338385" watchObservedRunningTime="2026-07-07 13:28:34.319967616 +0000 UTC m=+48.852873340"
Jul 07 13:28:34.958432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:34.958391    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:35.156190 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:35.156159    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:35.239218 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:35.239105    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:35.342251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:35.342211    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:35.440164 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:35.440113    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:35.539977 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:35.539878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:35.850954 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:35.850921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:36.443249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:36.443216    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:36.591108 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:36.591071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") pod \"image-registry-5486f79654-d4kzb\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") " pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:36.591283 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:36.591137    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:36.591283 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:36.591224    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:28:36.591283 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:36.591236    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:28:36.591283 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:36.591241    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5486f79654-d4kzb: secret "image-registry-tls" not found
Jul 07 13:28:36.591423 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:36.591289    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls podName:b3f1c729-7dac-4fc7-bd63-780b499660b9 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:52.59127615 +0000 UTC m=+67.124181853 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls") pod "dns-default-7lgc2" (UID: "b3f1c729-7dac-4fc7-bd63-780b499660b9") : secret "dns-default-metrics-tls" not found
Jul 07 13:28:36.591423 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:36.591302    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls podName:ad9a0f71-cf78-4021-90c7-f0d118473d98 nodeName:}" failed. No retries permitted until 2026-07-07 13:28:52.591296802 +0000 UTC m=+67.124202505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls") pod "image-registry-5486f79654-d4kzb" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98") : secret "image-registry-tls" not found
Jul 07 13:28:36.849616 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:36.849581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:37.246695 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:37.246600    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:37.440968 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:37.440938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:38.043695 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:38.043659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:38.385891 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:38.385820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:38.647929 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:38.647858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:38.717436 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:38.717404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:38.851668 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:38.851634    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:39.262088 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:39.262050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:39.405790 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:39.405727    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:39.491377 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:39.491342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:39.648898 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:39.648865    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:40.657882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:40.657850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:41.059398 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.059309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:41.249956 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.249898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:41.317097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.317001    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:41.455751 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.455713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:41.650294 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.650255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:41.828669 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.828627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:41.831074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.831042    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d61a68c1-2c75-40a8-9e32-0e903fbd9d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hl4gp\" (UID: \"d61a68c1-2c75-40a8-9e32-0e903fbd9d13\") " pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:41.909883 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.909793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hl4gp"
Jul 07 13:28:41.984975 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:41.984945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:42.037200 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:42.037171    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hl4gp"]
Jul 07 13:28:42.040649 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:42.040621    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61a68c1_2c75_40a8_9e32_0e903fbd9d13.slice/crio-5b1dea0f0f480aadca05fd960ddb7946dfb8e06fe5704bbfb810cd6eab36fcbd WatchSource:0}: Error finding container 5b1dea0f0f480aadca05fd960ddb7946dfb8e06fe5704bbfb810cd6eab36fcbd: Status 404 returned error can't find the container with id 5b1dea0f0f480aadca05fd960ddb7946dfb8e06fe5704bbfb810cd6eab36fcbd
Jul 07 13:28:42.324025 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:42.323942    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hl4gp" event={"ID":"d61a68c1-2c75-40a8-9e32-0e903fbd9d13","Type":"ContainerStarted","Data":"5b1dea0f0f480aadca05fd960ddb7946dfb8e06fe5704bbfb810cd6eab36fcbd"}
Jul 07 13:28:42.472195 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:42.472135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:42.650993 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:42.650956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:42.979159 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:42.978823    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:43.860393 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:43.860357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:44.044280 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:44.044195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:44.245431 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:44.245351    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:44.645469 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:44.645428    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:45.249270 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:45.249235    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:45.446130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:45.446092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:45.652252 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:45.652205    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:46.336598 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:46.336560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hl4gp" event={"ID":"d61a68c1-2c75-40a8-9e32-0e903fbd9d13","Type":"ContainerStarted","Data":"6bc2049b7b56b53b0eecee7e9adaad87516bc31d15201cf5432b1595f58611f0"}
Jul 07 13:28:47.263393 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:47.263363    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmv42"
Jul 07 13:28:47.341927 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:47.341891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hl4gp" event={"ID":"d61a68c1-2c75-40a8-9e32-0e903fbd9d13","Type":"ContainerStarted","Data":"8dd0c02c69f1efe2d389634205605aedf2b48d581d7311f0b1f11af7e5a326dd"}
Jul 07 13:28:48.031823 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:48.031788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:48.134401 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:48.134365    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:49.349444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:49.349411    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hl4gp" event={"ID":"d61a68c1-2c75-40a8-9e32-0e903fbd9d13","Type":"ContainerStarted","Data":"3f94953172f12f6de7b67ecd4461492978725dce036483b9fe5f4b3f01b53c8f"}
Jul 07 13:28:49.368912 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:49.368833    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hl4gp" podStartSLOduration=17.428444941 podStartE2EDuration="24.368817146s" totalImagesPullingTime="6.940372205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:28:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:28:49.367170287 +0000 UTC m=+63.900076022" watchObservedRunningTime="2026-07-07 13:28:49.368817146 +0000 UTC m=+63.901722870"
Jul 07 13:28:50.441130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.441096    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:50.542675 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.542629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:50.641051 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.641017    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:50.754403 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.754331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:50.916677 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.916647    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"]
Jul 07 13:28:50.918691 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.918676    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:50.921395 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.921365    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 13:28:50.921395 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.921383    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 13:28:50.922532 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.922515    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 13:28:50.922532 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.922523    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 07 13:28:50.928347 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.928329    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"]
Jul 07 13:28:50.941635 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.941614    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"]
Jul 07 13:28:50.943564 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.943548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:50.946083 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.946062    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 13:28:50.946169 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.946161    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-v5t9b\""
Jul 07 13:28:50.959037 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.959014    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"]
Jul 07 13:28:50.977660 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:50.977609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.004902 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.004842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c08988e-7a5d-45d5-99f4-512624f10368-tmp\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.005021 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.004968    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9c08988e-7a5d-45d5-99f4-512624f10368-klusterlet-config\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.005021 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.005005    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9270c3fb-2a85-43bd-b024-3ed894a7ca66-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm\" (UID: \"9270c3fb-2a85-43bd-b024-3ed894a7ca66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.005134 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.005034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltfz2\" (UniqueName: \"kubernetes.io/projected/9c08988e-7a5d-45d5-99f4-512624f10368-kube-api-access-ltfz2\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.005134 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.005089    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx4pk\" (UniqueName: \"kubernetes.io/projected/9270c3fb-2a85-43bd-b024-3ed894a7ca66-kube-api-access-lx4pk\") pod \"managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm\" (UID: \"9270c3fb-2a85-43bd-b024-3ed894a7ca66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.066471 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.066439    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5486f79654-d4kzb"]
Jul 07 13:28:51.066679 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:28:51.066662    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5486f79654-d4kzb" podUID="ad9a0f71-cf78-4021-90c7-f0d118473d98"
Jul 07 13:28:51.085582 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.085546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.106384 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.106356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c08988e-7a5d-45d5-99f4-512624f10368-tmp\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.106553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.106421    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9c08988e-7a5d-45d5-99f4-512624f10368-klusterlet-config\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.106553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.106438    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9270c3fb-2a85-43bd-b024-3ed894a7ca66-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm\" (UID: \"9270c3fb-2a85-43bd-b024-3ed894a7ca66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.106553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.106461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltfz2\" (UniqueName: \"kubernetes.io/projected/9c08988e-7a5d-45d5-99f4-512624f10368-kube-api-access-ltfz2\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.106553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.106515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx4pk\" (UniqueName: \"kubernetes.io/projected/9270c3fb-2a85-43bd-b024-3ed894a7ca66-kube-api-access-lx4pk\") pod \"managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm\" (UID: \"9270c3fb-2a85-43bd-b024-3ed894a7ca66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.106848 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.106830    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c08988e-7a5d-45d5-99f4-512624f10368-tmp\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.108876 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.108850    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9c08988e-7a5d-45d5-99f4-512624f10368-klusterlet-config\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.109088 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.109067    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9270c3fb-2a85-43bd-b024-3ed894a7ca66-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm\" (UID: \"9270c3fb-2a85-43bd-b024-3ed894a7ca66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.116470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.116448    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx4pk\" (UniqueName: \"kubernetes.io/projected/9270c3fb-2a85-43bd-b024-3ed894a7ca66-kube-api-access-lx4pk\") pod \"managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm\" (UID: \"9270c3fb-2a85-43bd-b024-3ed894a7ca66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.117112 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.117093    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltfz2\" (UniqueName: \"kubernetes.io/projected/9c08988e-7a5d-45d5-99f4-512624f10368-kube-api-access-ltfz2\") pod \"klusterlet-addon-workmgr-9474d5d9-gm88t\" (UID: \"9c08988e-7a5d-45d5-99f4-512624f10368\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.219294 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.219255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.228351 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.228325    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:28:51.263339 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.263276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"
Jul 07 13:28:51.308061 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.308028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.353956 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.353931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:51.361160 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.361025    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:51.362078 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.362052    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"]
Jul 07 13:28:51.365459 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:51.365437    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c08988e_7a5d_45d5_99f4_512624f10368.slice/crio-6bb5f9269be028b4d2dab21df25c2991b1152f54ea4eed8f581ae65fb483e1f9 WatchSource:0}: Error finding container 6bb5f9269be028b4d2dab21df25c2991b1152f54ea4eed8f581ae65fb483e1f9: Status 404 returned error can't find the container with id 6bb5f9269be028b4d2dab21df25c2991b1152f54ea4eed8f581ae65fb483e1f9
Jul 07 13:28:51.402352 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.402173    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm"]
Jul 07 13:28:51.406543 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.406517    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.406960 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:51.406939    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9270c3fb_2a85_43bd_b024_3ed894a7ca66.slice/crio-93d0493c444821acc15bd1b525fe7dd7307814863d15dbb76481dc34b1fd5484 WatchSource:0}: Error finding container 93d0493c444821acc15bd1b525fe7dd7307814863d15dbb76481dc34b1fd5484: Status 404 returned error can't find the container with id 93d0493c444821acc15bd1b525fe7dd7307814863d15dbb76481dc34b1fd5484
Jul 07 13:28:51.407849 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.407831    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb8k7\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-kube-api-access-nb8k7\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.407944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.407858    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-trusted-ca\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.407944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.407894    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-bound-sa-token\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.407944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.407919    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ad9a0f71-cf78-4021-90c7-f0d118473d98-ca-trust-extracted\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.407944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.407936    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-image-registry-private-configuration\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.408097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.407989    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-installation-pull-secrets\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.408097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.408013    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-certificates\") pod \"ad9a0f71-cf78-4021-90c7-f0d118473d98\" (UID: \"ad9a0f71-cf78-4021-90c7-f0d118473d98\") "
Jul 07 13:28:51.408351 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.408323    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:28:51.408430 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.408376    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:28:51.408430 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.408400    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9a0f71-cf78-4021-90c7-f0d118473d98-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:28:51.410528 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.410503    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-kube-api-access-nb8k7" (OuterVolumeSpecName: "kube-api-access-nb8k7") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "kube-api-access-nb8k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:28:51.410608 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.410536    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:28:51.410608 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.410555    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:28:51.410608 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.410564    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ad9a0f71-cf78-4021-90c7-f0d118473d98" (UID: "ad9a0f71-cf78-4021-90c7-f0d118473d98"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:28:51.508841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508805    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb8k7\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-kube-api-access-nb8k7\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.508841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508830    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-trusted-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.508841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508839    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-bound-sa-token\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.508841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508848    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ad9a0f71-cf78-4021-90c7-f0d118473d98-ca-trust-extracted\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.509350 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508857    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-image-registry-private-configuration\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.509350 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508867    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ad9a0f71-cf78-4021-90c7-f0d118473d98-installation-pull-secrets\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.509350 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.508875    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-certificates\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:51.509811 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.509769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.596581 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.596547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.692531 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.692496    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.710737 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.710707    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:51.713437 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.713421    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 13:28:51.723198 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.723180    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d32c03b-8bd2-4e94-83c6-d6c6a5927d47-metrics-certs\") pod \"network-metrics-daemon-sg4vv\" (UID: \"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47\") " pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:51.811664 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.811628    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:51.814586 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.814569    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 13:28:51.824398 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.824381    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8332404-cb66-4442-a5e4-59b4ea994ab7-original-pull-secret\") pod \"global-pull-secret-syncer-pjn5s\" (UID: \"f8332404-cb66-4442-a5e4-59b4ea994ab7\") " pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:51.912809 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.912693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:51.915629 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.915608    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 13:28:51.926541 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.926521    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 13:28:51.936885 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.936857    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4bjh\" (UniqueName: \"kubernetes.io/projected/580dbd22-63bc-42c6-b087-f118b618e48d-kube-api-access-v4bjh\") pod \"network-check-target-ttjh4\" (UID: \"580dbd22-63bc-42c6-b087-f118b618e48d\") " pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:51.980372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.980339    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:51.987652 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.987635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k6stv\""
Jul 07 13:28:51.993191 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.993157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjn5s"
Jul 07 13:28:51.996109 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:51.995984    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg4vv"
Jul 07 13:28:52.010591 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.010571    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4z824\""
Jul 07 13:28:52.019315 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.019046    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:28:52.097845 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.096299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:52.187843 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.187722    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pjn5s"]
Jul 07 13:28:52.188005 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.187979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:52.195505 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:52.193920    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8332404_cb66_4442_a5e4_59b4ea994ab7.slice/crio-210e8dfbc2a3e8221d5b48a6ba22c7a9059ff3f7940ed945664b95e3765d0d2f WatchSource:0}: Error finding container 210e8dfbc2a3e8221d5b48a6ba22c7a9059ff3f7940ed945664b95e3765d0d2f: Status 404 returned error can't find the container with id 210e8dfbc2a3e8221d5b48a6ba22c7a9059ff3f7940ed945664b95e3765d0d2f
Jul 07 13:28:52.221714 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.221687    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sg4vv"]
Jul 07 13:28:52.254325 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.254252    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ttjh4"]
Jul 07 13:28:52.258453 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:52.258396    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod580dbd22_63bc_42c6_b087_f118b618e48d.slice/crio-e2739079c6adbfb074f03c02878d7e2f2b0e09f5383ee53430483c0e10f98b7e WatchSource:0}: Error finding container e2739079c6adbfb074f03c02878d7e2f2b0e09f5383ee53430483c0e10f98b7e: Status 404 returned error can't find the container with id e2739079c6adbfb074f03c02878d7e2f2b0e09f5383ee53430483c0e10f98b7e
Jul 07 13:28:52.359263 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.359192    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ttjh4" event={"ID":"580dbd22-63bc-42c6-b087-f118b618e48d","Type":"ContainerStarted","Data":"e2739079c6adbfb074f03c02878d7e2f2b0e09f5383ee53430483c0e10f98b7e"}
Jul 07 13:28:52.360743 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.360713    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sg4vv" event={"ID":"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47","Type":"ContainerStarted","Data":"eb7ee1c7a046dc61af84b31907faa9c75c4eae01109bd04ced19cd547324cc00"}
Jul 07 13:28:52.362028 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.362002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm" event={"ID":"9270c3fb-2a85-43bd-b024-3ed894a7ca66","Type":"ContainerStarted","Data":"93d0493c444821acc15bd1b525fe7dd7307814863d15dbb76481dc34b1fd5484"}
Jul 07 13:28:52.363729 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.363706    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t" event={"ID":"9c08988e-7a5d-45d5-99f4-512624f10368","Type":"ContainerStarted","Data":"6bb5f9269be028b4d2dab21df25c2991b1152f54ea4eed8f581ae65fb483e1f9"}
Jul 07 13:28:52.365159 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.365139    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5486f79654-d4kzb"
Jul 07 13:28:52.365878 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.365624    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pjn5s" event={"ID":"f8332404-cb66-4442-a5e4-59b4ea994ab7","Type":"ContainerStarted","Data":"210e8dfbc2a3e8221d5b48a6ba22c7a9059ff3f7940ed945664b95e3765d0d2f"}
Jul 07 13:28:52.407880 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.407135    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5486f79654-d4kzb"]
Jul 07 13:28:52.419010 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.418952    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5486f79654-d4kzb"]
Jul 07 13:28:52.518620 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.518541    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ad9a0f71-cf78-4021-90c7-f0d118473d98-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:28:52.620322 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.619829    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:52.623599 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.623498    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3f1c729-7dac-4fc7-bd63-780b499660b9-metrics-tls\") pod \"dns-default-7lgc2\" (UID: \"b3f1c729-7dac-4fc7-bd63-780b499660b9\") " pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:52.881595 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.881354    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rgbgq\""
Jul 07 13:28:52.891277 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:52.891048    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:28:53.126333 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:53.118158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lgc2"]
Jul 07 13:28:53.134708 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:53.134640    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f1c729_7dac_4fc7_bd63_780b499660b9.slice/crio-105e8caa4b46d1ac21af927487e9899f28e7f5099d34eaf02a702827048b6f3f WatchSource:0}: Error finding container 105e8caa4b46d1ac21af927487e9899f28e7f5099d34eaf02a702827048b6f3f: Status 404 returned error can't find the container with id 105e8caa4b46d1ac21af927487e9899f28e7f5099d34eaf02a702827048b6f3f
Jul 07 13:28:53.151033 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:53.150996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:53.371670 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:53.371633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lgc2" event={"ID":"b3f1c729-7dac-4fc7-bd63-780b499660b9","Type":"ContainerStarted","Data":"105e8caa4b46d1ac21af927487e9899f28e7f5099d34eaf02a702827048b6f3f"}
Jul 07 13:28:54.078731 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.078543    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad9a0f71-cf78-4021-90c7-f0d118473d98" path="/var/lib/kubelet/pods/ad9a0f71-cf78-4021-90c7-f0d118473d98/volumes"
Jul 07 13:28:54.184932 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.184820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:54.274909 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.274854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:54.276873 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.276848    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"]
Jul 07 13:28:54.331048 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.330972    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"]
Jul 07 13:28:54.331211 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.331101    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:28:54.334114 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.334092    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 07 13:28:54.334352 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.334334    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kr99n\""
Jul 07 13:28:54.353513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.353422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:54.439472 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.439436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c0ee24ed-745a-4757-b435-00c8ac0d2689-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-njnlp\" (UID: \"c0ee24ed-745a-4757-b435-00c8ac0d2689\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:28:54.457608 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.457576    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:28:54.541061 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.540686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c0ee24ed-745a-4757-b435-00c8ac0d2689-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-njnlp\" (UID: \"c0ee24ed-745a-4757-b435-00c8ac0d2689\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:28:54.549076 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.549045    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c0ee24ed-745a-4757-b435-00c8ac0d2689-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-njnlp\" (UID: \"c0ee24ed-745a-4757-b435-00c8ac0d2689\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:28:54.645923 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:54.645641    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:28:59.825611 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:28:59.825567    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"]
Jul 07 13:28:59.831710 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:28:59.831668    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ee24ed_745a_4757_b435_00c8ac0d2689.slice/crio-e1fcee843e001e82be663c41c52f40884c528d63fe28d4a92ba8de3835380180 WatchSource:0}: Error finding container e1fcee843e001e82be663c41c52f40884c528d63fe28d4a92ba8de3835380180: Status 404 returned error can't find the container with id e1fcee843e001e82be663c41c52f40884c528d63fe28d4a92ba8de3835380180
Jul 07 13:29:00.179708 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.179616    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:00.274336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.274298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:00.378465 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.378425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:00.395130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.395098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ttjh4" event={"ID":"580dbd22-63bc-42c6-b087-f118b618e48d","Type":"ContainerStarted","Data":"22e68d14f911293c96462d2ce918015dfdd3fc248d9511786416c8a25ff37c19"}
Jul 07 13:29:00.395280 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.395170    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:29:00.396826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.396800    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lgc2" event={"ID":"b3f1c729-7dac-4fc7-bd63-780b499660b9","Type":"ContainerStarted","Data":"381e4c1fb9cfd6b7ee84d5250e5f551361e5431c7dbe3923d6286c3829f01eed"}
Jul 07 13:29:00.396944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.396840    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lgc2" event={"ID":"b3f1c729-7dac-4fc7-bd63-780b499660b9","Type":"ContainerStarted","Data":"93ea4c3f985f7d07a63573d8b38dbd89af8b1cbca3e96e43904911eac1f0d5f9"}
Jul 07 13:29:00.396944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.396865    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:29:00.397909 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.397888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp" event={"ID":"c0ee24ed-745a-4757-b435-00c8ac0d2689","Type":"ContainerStarted","Data":"e1fcee843e001e82be663c41c52f40884c528d63fe28d4a92ba8de3835380180"}
Jul 07 13:29:00.399479 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.399458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sg4vv" event={"ID":"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47","Type":"ContainerStarted","Data":"73e127cc3704ba9af663f99d84b69e12ad537c6c2f545fa129fdd38c4f0ef589"}
Jul 07 13:29:00.399581 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.399503    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sg4vv" event={"ID":"4d32c03b-8bd2-4e94-83c6-d6c6a5927d47","Type":"ContainerStarted","Data":"913662b32b9be569e57c8ee7a68d3144e9d18312fa38c7e218b4e393e64b46e4"}
Jul 07 13:29:00.400713 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.400689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm" event={"ID":"9270c3fb-2a85-43bd-b024-3ed894a7ca66","Type":"ContainerStarted","Data":"09e494472932bfca0d363c81964aaf48ce18d5cfcab3b426b4ff7c3792944863"}
Jul 07 13:29:00.401994 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.401974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t" event={"ID":"9c08988e-7a5d-45d5-99f4-512624f10368","Type":"ContainerStarted","Data":"c0a57c2e16d93f6c65b7e6c0c022b782923d9de88d9f04b9d31ff6abe1b9d549"}
Jul 07 13:29:00.402095 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.402076    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:29:00.403415 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.403396    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pjn5s" event={"ID":"f8332404-cb66-4442-a5e4-59b4ea994ab7","Type":"ContainerStarted","Data":"3f4b2eb5ea0c4b59a5c1a60323270fe7075eaa5827595855f81a464fb285ab51"}
Jul 07 13:29:00.404034 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.404020    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t"
Jul 07 13:29:00.441391 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.441093    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ttjh4" podStartSLOduration=67.01399482 podStartE2EDuration="1m14.441074676s" totalImagesPullingTime="7.427079856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:00.438402046 +0000 UTC m=+74.971307801" watchObservedRunningTime="2026-07-07 13:29:00.441074676 +0000 UTC m=+74.973980381"
Jul 07 13:29:00.464101 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.464061    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d8f5b947b-sgjzm" podStartSLOduration=2.202373446 podStartE2EDuration="10.464041999s" totalImagesPullingTime="8.261668553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:00.463478382 +0000 UTC m=+74.996384101" watchObservedRunningTime="2026-07-07 13:29:00.464041999 +0000 UTC m=+74.996947726"
Jul 07 13:29:00.486073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.485743    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7lgc2" podStartSLOduration=33.918060296 podStartE2EDuration="40.485728386s" totalImagesPullingTime="6.56766809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:00.485437071 +0000 UTC m=+75.018342826" watchObservedRunningTime="2026-07-07 13:29:00.485728386 +0000 UTC m=+75.018634111"
Jul 07 13:29:00.506120 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.506070    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9474d5d9-gm88t" podStartSLOduration=2.202597793 podStartE2EDuration="10.506051366s" totalImagesPullingTime="8.303453573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:00.504672315 +0000 UTC m=+75.037578041" watchObservedRunningTime="2026-07-07 13:29:00.506051366 +0000 UTC m=+75.038957092"
Jul 07 13:29:00.523174 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.523128    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sg4vv" podStartSLOduration=67.552120099 podStartE2EDuration="1m14.523109737s" totalImagesPullingTime="6.970989638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:00.52179419 +0000 UTC m=+75.054699927" watchObservedRunningTime="2026-07-07 13:29:00.523109737 +0000 UTC m=+75.056015463"
Jul 07 13:29:00.546199 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.546147    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pjn5s" podStartSLOduration=67.058636822 podStartE2EDuration="1m14.546128603s" totalImagesPullingTime="7.487491781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:00.544827722 +0000 UTC m=+75.077733479" watchObservedRunningTime="2026-07-07 13:29:00.546128603 +0000 UTC m=+75.079034330"
Jul 07 13:29:00.642725 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.642694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:00.746123 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.746023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:00.846898 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.846867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:00.946973 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:00.946931    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:01.408480 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:01.408444    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp" event={"ID":"c0ee24ed-745a-4757-b435-00c8ac0d2689","Type":"ContainerStarted","Data":"74c77e1d51437d518d24a01690fc60670ad76b22fba7be4ab3181ede8ac21cc1"}
Jul 07 13:29:01.408947 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:01.408898    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:29:01.413157 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:01.413141    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp"
Jul 07 13:29:01.428798 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:01.428733    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-njnlp" podStartSLOduration=6.458287591 podStartE2EDuration="7.428718126s" totalImagesPullingTime="970.430535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:01.428013449 +0000 UTC m=+75.960919188" watchObservedRunningTime="2026-07-07 13:29:01.428718126 +0000 UTC m=+75.961623852"
Jul 07 13:29:02.349028 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.348987    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:02.415107 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.415081    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"]
Jul 07 13:29:02.417202 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.417188    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.420997 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.420970    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 07 13:29:02.421137 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.421115    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 13:29:02.421208 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.421120    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 07 13:29:02.421266 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.420971    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 13:29:02.421266 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.421242    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r78ft\""
Jul 07 13:29:02.421266 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.421124    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 13:29:02.429211 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.429191    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"]
Jul 07 13:29:02.438706 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.438673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:02.500649 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.500613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea3c1f24-ec99-42b1-9866-05095d0e406f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.500649 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.500655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x46p\" (UniqueName: \"kubernetes.io/projected/ea3c1f24-ec99-42b1-9866-05095d0e406f-kube-api-access-2x46p\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.500907 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.500676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea3c1f24-ec99-42b1-9866-05095d0e406f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.500907 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.500846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea3c1f24-ec99-42b1-9866-05095d0e406f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.511558 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.511529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:02.601675 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.601607    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea3c1f24-ec99-42b1-9866-05095d0e406f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.601675 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.601652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea3c1f24-ec99-42b1-9866-05095d0e406f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.601863 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.601684    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x46p\" (UniqueName: \"kubernetes.io/projected/ea3c1f24-ec99-42b1-9866-05095d0e406f-kube-api-access-2x46p\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.601863 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.601704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea3c1f24-ec99-42b1-9866-05095d0e406f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.602429 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.602405    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea3c1f24-ec99-42b1-9866-05095d0e406f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.604010 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.603983    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea3c1f24-ec99-42b1-9866-05095d0e406f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.604010 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.603991    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea3c1f24-ec99-42b1-9866-05095d0e406f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.613889 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.613867    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x46p\" (UniqueName: \"kubernetes.io/projected/ea3c1f24-ec99-42b1-9866-05095d0e406f-kube-api-access-2x46p\") pod \"prometheus-operator-567f7b8b7c-9h48r\" (UID: \"ea3c1f24-ec99-42b1-9866-05095d0e406f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.726461 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.726423    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"
Jul 07 13:29:02.869819 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:02.869732    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r"]
Jul 07 13:29:02.873149 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:02.873124    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3c1f24_ec99_42b1_9866_05095d0e406f.slice/crio-f97cdcd2a60fb13be597d0af9ddd645fb7f664818f61c1ca4eebab6283fa4c8f WatchSource:0}: Error finding container f97cdcd2a60fb13be597d0af9ddd645fb7f664818f61c1ca4eebab6283fa4c8f: Status 404 returned error can't find the container with id f97cdcd2a60fb13be597d0af9ddd645fb7f664818f61c1ca4eebab6283fa4c8f
Jul 07 13:29:03.419481 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:03.419445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r" event={"ID":"ea3c1f24-ec99-42b1-9866-05095d0e406f","Type":"ContainerStarted","Data":"f97cdcd2a60fb13be597d0af9ddd645fb7f664818f61c1ca4eebab6283fa4c8f"}
Jul 07 13:29:03.539218 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:03.539178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:04.366368 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:04.366323    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:04.424731 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:04.424696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r" event={"ID":"ea3c1f24-ec99-42b1-9866-05095d0e406f","Type":"ContainerStarted","Data":"e8c0e56c3018bc70eaa3464da271cde6724fd4615c86972d4479dee2c41007bd"}
Jul 07 13:29:04.425115 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:04.424744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r" event={"ID":"ea3c1f24-ec99-42b1-9866-05095d0e406f","Type":"ContainerStarted","Data":"60e30300859367f04b45e40b352e7729b4b717e471aa458723f9ad29090cdd6d"}
Jul 07 13:29:04.443820 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:04.443679    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9h48r" podStartSLOduration=1.226977975 podStartE2EDuration="2.44366044s" totalImagesPullingTime="1.216682465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:04.442880726 +0000 UTC m=+78.975786472" watchObservedRunningTime="2026-07-07 13:29:04.44366044 +0000 UTC m=+78.976566166"
Jul 07 13:29:06.808679 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.808646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:06.840316 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.840287    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"]
Jul 07 13:29:06.843066 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.843049    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:06.850046 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.850024    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-275z7"]
Jul 07 13:29:06.852821 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.852802    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.854815 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.854794    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"]
Jul 07 13:29:06.857344 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.857327    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.859095 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.859079    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 07 13:29:06.859366 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.859349    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 07 13:29:06.859366 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.859352    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-6z2kr\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6z2kr\"" type="*v1.Secret"
Jul 07 13:29:06.860034 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.860008    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap"
Jul 07 13:29:06.860721 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.860696    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xdhr8\""
Jul 07 13:29:06.860887 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.860813    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret"
Jul 07 13:29:06.860968 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.860904    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-dockercfg-9rzbc\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9rzbc\"" type="*v1.Secret"
Jul 07 13:29:06.861038 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.860997    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-tls\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" type="*v1.Secret"
Jul 07 13:29:06.861174 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.861149    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret"
Jul 07 13:29:06.861544 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.861512    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-state-metrics-custom-resource-state-configmap\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" type="*v1.ConfigMap"
Jul 07 13:29:06.863985 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:06.863956    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret"
Jul 07 13:29:06.912259 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.912207    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"]
Jul 07 13:29:06.932688 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.932688 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9nr\" (UniqueName: \"kubernetes.io/projected/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-kube-api-access-7l9nr\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:06.932882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932710    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-sys\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.932882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-textfile\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.932882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:06.932882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:06.932882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/928a877b-4cbb-4a66-b732-9309e90cfd0e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.932882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-root\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.932997    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/928a877b-4cbb-4a66-b732-9309e90cfd0e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-tls\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933052    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b398a611-22c3-4ec9-a0bb-5395e6726d66-metrics-client-ca\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.933130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933125    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjrl\" (UniqueName: \"kubernetes.io/projected/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-api-access-hdjrl\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:06.933389 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933154    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hghhz\" (UniqueName: \"kubernetes.io/projected/b398a611-22c3-4ec9-a0bb-5395e6726d66-kube-api-access-hghhz\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.933389 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-wtmp\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.933389 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.933223    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-accelerators-collector-config\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:06.970269 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.970224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:06.972549 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:06.970869    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"]
Jul 07 13:29:07.034045 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/928a877b-4cbb-4a66-b732-9309e90cfd0e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034045 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034304 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-root\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034304 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034105    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-root\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034304 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.034304 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/928a877b-4cbb-4a66-b732-9309e90cfd0e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034494 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-tls\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034494 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b398a611-22c3-4ec9-a0bb-5395e6726d66-metrics-client-ca\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034494 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034648 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034648 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034555    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjrl\" (UniqueName: \"kubernetes.io/projected/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-api-access-hdjrl\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034648 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hghhz\" (UniqueName: \"kubernetes.io/projected/b398a611-22c3-4ec9-a0bb-5395e6726d66-kube-api-access-hghhz\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034648 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-wtmp\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-accelerators-collector-config\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034681    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/928a877b-4cbb-4a66-b732-9309e90cfd0e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034697    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034739    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9nr\" (UniqueName: \"kubernetes.io/projected/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-kube-api-access-7l9nr\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-sys\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-textfile\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034828    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/928a877b-4cbb-4a66-b732-9309e90cfd0e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034837    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-wtmp\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.034857 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.035272 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.035272 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:07.034916    2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 07 13:29:07.035272 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:07.034971    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-tls podName:77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7 nodeName:}" failed. No retries permitted until 2026-07-07 13:29:07.534953378 +0000 UTC m=+82.067859081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-mthk7" (UID: "77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7") : secret "openshift-state-metrics-tls" not found
Jul 07 13:29:07.035272 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034978    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b398a611-22c3-4ec9-a0bb-5395e6726d66-metrics-client-ca\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.035272 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.034978    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b398a611-22c3-4ec9-a0bb-5395e6726d66-sys\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.035272 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.035189    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-textfile\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.035459 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.035424    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.036731 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.036712    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.052112 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.052090    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjrl\" (UniqueName: \"kubernetes.io/projected/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-api-access-hdjrl\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:07.052612 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.052596    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9nr\" (UniqueName: \"kubernetes.io/projected/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-kube-api-access-7l9nr\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.059358 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.059302    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hghhz\" (UniqueName: \"kubernetes.io/projected/b398a611-22c3-4ec9-a0bb-5395e6726d66-kube-api-access-hghhz\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.061924 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.061899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.157221 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.157188    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.257945 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.257908    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.355995 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.355964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.492493 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.492457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.537758 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.537724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.540084 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.540062    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-mthk7\" (UID: \"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.737408 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.737323    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9rzbc\""
Jul 07 13:29:07.753395 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.753363    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.754130 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.754114    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"
Jul 07 13:29:07.818817 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.818791    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6z2kr\""
Jul 07 13:29:07.848693 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.848648    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.900700 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.900672    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7"]
Jul 07 13:29:07.903583 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:07.903557    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77825edc_aeb8_44aa_8fd9_d0fd69c3dcb7.slice/crio-37a6c8305b5e9fdeec074b26d43f79060f4b5847afd61395272b96c05738edb0 WatchSource:0}: Error finding container 37a6c8305b5e9fdeec074b26d43f79060f4b5847afd61395272b96c05738edb0: Status 404 returned error can't find the container with id 37a6c8305b5e9fdeec074b26d43f79060f4b5847afd61395272b96c05738edb0
Jul 07 13:29:07.930547 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.930524    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 13:29:07.934929 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.934904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:07.937024 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.937006    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:07.973895 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.973871    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 07 13:29:07.977261 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:07.977239    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:08.010931 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.010909    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 07 13:29:08.013345 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.013322    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 07 13:29:08.015340 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.015315    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:08.015626 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.015610    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 13:29:08.017032 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.017012    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/928a877b-4cbb-4a66-b732-9309e90cfd0e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8rrlx\" (UID: \"928a877b-4cbb-4a66-b732-9309e90cfd0e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:08.025980 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.025960    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-accelerators-collector-config\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:08.034523 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:08.034503    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: failed to sync secret cache: timed out waiting for the condition
Jul 07 13:29:08.034615 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:08.034577    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-tls podName:b398a611-22c3-4ec9-a0bb-5395e6726d66 nodeName:}" failed. No retries permitted until 2026-07-07 13:29:08.534557863 +0000 UTC m=+83.067463582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-tls") pod "node-exporter-275z7" (UID: "b398a611-22c3-4ec9-a0bb-5395e6726d66") : failed to sync secret cache: timed out waiting for the condition
Jul 07 13:29:08.071983 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.071956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"
Jul 07 13:29:08.204185 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.204164    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx"]
Jul 07 13:29:08.206881 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:08.206852    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928a877b_4cbb_4a66_b732_9309e90cfd0e.slice/crio-0b279d0ebc8c346dedc13d1ad040e5ac42205aaa059ec704883c90fc6bd085d5 WatchSource:0}: Error finding container 0b279d0ebc8c346dedc13d1ad040e5ac42205aaa059ec704883c90fc6bd085d5: Status 404 returned error can't find the container with id 0b279d0ebc8c346dedc13d1ad040e5ac42205aaa059ec704883c90fc6bd085d5
Jul 07 13:29:08.362282 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.362256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 13:29:08.441253 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.441219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx" event={"ID":"928a877b-4cbb-4a66-b732-9309e90cfd0e","Type":"ContainerStarted","Data":"0b279d0ebc8c346dedc13d1ad040e5ac42205aaa059ec704883c90fc6bd085d5"}
Jul 07 13:29:08.443009 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.442973    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7" event={"ID":"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7","Type":"ContainerStarted","Data":"e961267e21ee73387e596c4062975fd1268be7877c307590cd59ea849f939641"}
Jul 07 13:29:08.443125 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.443022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7" event={"ID":"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7","Type":"ContainerStarted","Data":"7b3e9c7f7ec61590d8c8a0da2a4f2166fe3aa9cc6cffd27c42d16f04ca5be516"}
Jul 07 13:29:08.443125 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.443043    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7" event={"ID":"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7","Type":"ContainerStarted","Data":"37a6c8305b5e9fdeec074b26d43f79060f4b5847afd61395272b96c05738edb0"}
Jul 07 13:29:08.464603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.464572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:08.546649 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.546620    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-tls\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:08.548886 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.548869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b398a611-22c3-4ec9-a0bb-5395e6726d66-node-exporter-tls\") pod \"node-exporter-275z7\" (UID: \"b398a611-22c3-4ec9-a0bb-5395e6726d66\") " pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:08.665279 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:08.665197    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-275z7"
Jul 07 13:29:08.676081 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:08.676057    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb398a611_22c3_4ec9_a0bb_5395e6726d66.slice/crio-3f8cfce964c9e0c1baf6138ac2e9008ef64ffd2ff55b19325538df5a6e3c6d06 WatchSource:0}: Error finding container 3f8cfce964c9e0c1baf6138ac2e9008ef64ffd2ff55b19325538df5a6e3c6d06: Status 404 returned error can't find the container with id 3f8cfce964c9e0c1baf6138ac2e9008ef64ffd2ff55b19325538df5a6e3c6d06
Jul 07 13:29:09.451570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:09.450962    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7" event={"ID":"77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7","Type":"ContainerStarted","Data":"be9ac8382b8c7962c16383c2be1eb30ebb6eb229ab3cba1c0fbbd149c21cff26"}
Jul 07 13:29:09.452990 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:09.452967    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-275z7" event={"ID":"b398a611-22c3-4ec9-a0bb-5395e6726d66","Type":"ContainerStarted","Data":"3f8cfce964c9e0c1baf6138ac2e9008ef64ffd2ff55b19325538df5a6e3c6d06"}
Jul 07 13:29:09.469492 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:09.469444    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-mthk7" podStartSLOduration=2.398859502 podStartE2EDuration="3.46942663s" totalImagesPullingTime="1.070567128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:09.468250353 +0000 UTC m=+84.001156104" watchObservedRunningTime="2026-07-07 13:29:09.46942663 +0000 UTC m=+84.002332355"
Jul 07 13:29:09.637662 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:09.637602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:09.742180 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:09.742091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:09.840420 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:09.840381    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:10.342604 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.342512    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7456b586d5-lk78f"]
Jul 07 13:29:10.345486 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.345462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.348887 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.348866    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 13:29:10.350074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.350034    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 13:29:10.350584 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.350565    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 13:29:10.350837 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.350817    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 13:29:10.351725 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.351555    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4l4hj\""
Jul 07 13:29:10.351829 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.351746    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 13:29:10.352829 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.352489    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 13:29:10.352829 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.352695    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 13:29:10.360964 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.360923    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 13:29:10.367575 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.367507    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7456b586d5-lk78f"]
Jul 07 13:29:10.411526 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.411490    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7lgc2"
Jul 07 13:29:10.459783 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.459733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx" event={"ID":"928a877b-4cbb-4a66-b732-9309e90cfd0e","Type":"ContainerStarted","Data":"1127d185527f0965543c44cebcd98b379607b131f99cb690b132c77e493f1895"}
Jul 07 13:29:10.460346 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.460323    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx" event={"ID":"928a877b-4cbb-4a66-b732-9309e90cfd0e","Type":"ContainerStarted","Data":"e1f2706e355551463093dc702d21d7f18f6af6d8342d5fb44bcafcb1bea71bbf"}
Jul 07 13:29:10.460502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.460472    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx" event={"ID":"928a877b-4cbb-4a66-b732-9309e90cfd0e","Type":"ContainerStarted","Data":"4edc46c2a89f85e84caf2b286496a4a963552093d17ffd09d86fc45b76196f32"}
Jul 07 13:29:10.462219 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.462092    2582 generic.go:356] "Generic (PLEG): container finished" podID="b398a611-22c3-4ec9-a0bb-5395e6726d66" containerID="75d915281904b1f3135afcce602d413e8b8df0ef279f6a14e217325cc977ef11" exitCode=0
Jul 07 13:29:10.462337 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.462210    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-275z7" event={"ID":"b398a611-22c3-4ec9-a0bb-5395e6726d66","Type":"ContainerDied","Data":"75d915281904b1f3135afcce602d413e8b8df0ef279f6a14e217325cc977ef11"}
Jul 07 13:29:10.463502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-trusted-ca-bundle\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.463502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtktv\" (UniqueName: \"kubernetes.io/projected/345e549a-20aa-4026-ab46-79bd610d2c31-kube-api-access-qtktv\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.463502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463304    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-oauth-serving-cert\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.463502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-oauth-config\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.463502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-serving-cert\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.463502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-service-ca\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.463830 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.463507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-console-config\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.484924 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.484869    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8rrlx" podStartSLOduration=2.7997383080000002 podStartE2EDuration="4.484849751s" totalImagesPullingTime="1.685111443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:10.483552024 +0000 UTC m=+85.016457749" watchObservedRunningTime="2026-07-07 13:29:10.484849751 +0000 UTC m=+85.017755478"
Jul 07 13:29:10.489767 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.489742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:10.563908 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.563880    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-oauth-config\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.564212 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.564186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-serving-cert\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.564298 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.564214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-service-ca\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.564298 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.564262    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-console-config\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.564390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.564318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-trusted-ca-bundle\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.564390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.564363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtktv\" (UniqueName: \"kubernetes.io/projected/345e549a-20aa-4026-ab46-79bd610d2c31-kube-api-access-qtktv\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.564499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.564416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-oauth-serving-cert\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.565157 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.565137    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-oauth-serving-cert\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.565496 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.565471    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-console-config\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.565592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.565546    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-service-ca\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.565659 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.565635    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-trusted-ca-bundle\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.566815 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.566792    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-oauth-config\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.567206 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.567187    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-serving-cert\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.573267 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.573240    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtktv\" (UniqueName: \"kubernetes.io/projected/345e549a-20aa-4026-ab46-79bd610d2c31-kube-api-access-qtktv\") pod \"console-7456b586d5-lk78f\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") " pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.585673 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.585647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:10.658591 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.658553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:10.785215 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:10.785187    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7456b586d5-lk78f"]
Jul 07 13:29:10.787668 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:10.787642    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345e549a_20aa_4026_ab46_79bd610d2c31.slice/crio-4c92985f2c34f73d8f6314a0d2eac97d80c1973455ac26f370d2321c79fdfc40 WatchSource:0}: Error finding container 4c92985f2c34f73d8f6314a0d2eac97d80c1973455ac26f370d2321c79fdfc40: Status 404 returned error can't find the container with id 4c92985f2c34f73d8f6314a0d2eac97d80c1973455ac26f370d2321c79fdfc40
Jul 07 13:29:11.283177 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.283143    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-796f687785-gfk4v"]
Jul 07 13:29:11.286204 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.286189    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.288865 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.288845    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 13:29:11.288978 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.288872    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dsdjg\""
Jul 07 13:29:11.289874 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.289846    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 13:29:11.290032 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.290013    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-762sqnaamobs5\""
Jul 07 13:29:11.290132 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.290018    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 13:29:11.290132 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.290087    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 13:29:11.295725 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.295705    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-796f687785-gfk4v"]
Jul 07 13:29:11.371441 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-secret-metrics-server-client-certs\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.371441 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99c88269-9e09-47c2-a89e-0f87a136de78-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.371651 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-client-ca-bundle\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.371651 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv4wq\" (UniqueName: \"kubernetes.io/projected/99c88269-9e09-47c2-a89e-0f87a136de78-kube-api-access-gv4wq\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.371651 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371528    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-secret-metrics-server-tls\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.371651 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/99c88269-9e09-47c2-a89e-0f87a136de78-audit-log\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.371651 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.371613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/99c88269-9e09-47c2-a89e-0f87a136de78-metrics-server-audit-profiles\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.392087 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.392058    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:11.472038 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.471996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/99c88269-9e09-47c2-a89e-0f87a136de78-audit-log\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/99c88269-9e09-47c2-a89e-0f87a136de78-metrics-server-audit-profiles\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-secret-metrics-server-client-certs\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99c88269-9e09-47c2-a89e-0f87a136de78-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-client-ca-bundle\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv4wq\" (UniqueName: \"kubernetes.io/projected/99c88269-9e09-47c2-a89e-0f87a136de78-kube-api-access-gv4wq\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-secret-metrics-server-tls\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.472467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.472452    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/99c88269-9e09-47c2-a89e-0f87a136de78-audit-log\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.473487 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.473455    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99c88269-9e09-47c2-a89e-0f87a136de78-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.473790 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.473725    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/99c88269-9e09-47c2-a89e-0f87a136de78-metrics-server-audit-profiles\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.475614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.475268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-secret-metrics-server-tls\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.475614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.475302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-275z7" event={"ID":"b398a611-22c3-4ec9-a0bb-5395e6726d66","Type":"ContainerStarted","Data":"2eecbe9fdfb7177fc59d6acfef9be2733756eab51e27c840ce855ef3c93d7408"}
Jul 07 13:29:11.475614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.475393    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-275z7" event={"ID":"b398a611-22c3-4ec9-a0bb-5395e6726d66","Type":"ContainerStarted","Data":"3fc63af380fbc1006eb1399c29129ea62fca14149c63d0171a1a028bedde6319"}
Jul 07 13:29:11.475614 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.475546    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-client-ca-bundle\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.476332 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.476303    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/99c88269-9e09-47c2-a89e-0f87a136de78-secret-metrics-server-client-certs\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.477010 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.476990    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456b586d5-lk78f" event={"ID":"345e549a-20aa-4026-ab46-79bd610d2c31","Type":"ContainerStarted","Data":"4c92985f2c34f73d8f6314a0d2eac97d80c1973455ac26f370d2321c79fdfc40"}
Jul 07 13:29:11.482461 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.482439    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv4wq\" (UniqueName: \"kubernetes.io/projected/99c88269-9e09-47c2-a89e-0f87a136de78-kube-api-access-gv4wq\") pod \"metrics-server-796f687785-gfk4v\" (UID: \"99c88269-9e09-47c2-a89e-0f87a136de78\") " pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.568802 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.568695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:11.575690 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.575650    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-275z7" podStartSLOduration=4.242732162 podStartE2EDuration="5.575637059s" totalImagesPullingTime="1.218480487s" totalInitContainerRuntime="114.42441ms" isStatefulPod=true podCreationTimestamp="2026-07-07 13:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:11.501455288 +0000 UTC m=+86.034361038" watchObservedRunningTime="2026-07-07 13:29:11.575637059 +0000 UTC m=+86.108542791"
Jul 07 13:29:11.576224 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.576200    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"]
Jul 07 13:29:11.579372 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.579358    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.582944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.582906    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 13:29:11.582944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.582941    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rh574\""
Jul 07 13:29:11.589593 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.589560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"]
Jul 07 13:29:11.596886 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.596723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:11.662073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.661965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:11.674611 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.674432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj58j\" (UniqueName: \"kubernetes.io/projected/05f8c282-030f-4b3a-bbf3-58cbadb98560-kube-api-access-gj58j\") pod \"monitoring-plugin-d5dcff7cf-j6xqr\" (UID: \"05f8c282-030f-4b3a-bbf3-58cbadb98560\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.674611 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.674532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05f8c282-030f-4b3a-bbf3-58cbadb98560-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-j6xqr\" (UID: \"05f8c282-030f-4b3a-bbf3-58cbadb98560\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.753852 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.753823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-796f687785-gfk4v"]
Jul 07 13:29:11.756825 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.756763    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:11.757933 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:11.757908    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c88269_9e09_47c2_a89e_0f87a136de78.slice/crio-f823a31e6247ab89281a3eca213637dc00d4c6198b40340aae12854f05b4d94f WatchSource:0}: Error finding container f823a31e6247ab89281a3eca213637dc00d4c6198b40340aae12854f05b4d94f: Status 404 returned error can't find the container with id f823a31e6247ab89281a3eca213637dc00d4c6198b40340aae12854f05b4d94f
Jul 07 13:29:11.775251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.775226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj58j\" (UniqueName: \"kubernetes.io/projected/05f8c282-030f-4b3a-bbf3-58cbadb98560-kube-api-access-gj58j\") pod \"monitoring-plugin-d5dcff7cf-j6xqr\" (UID: \"05f8c282-030f-4b3a-bbf3-58cbadb98560\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.775363 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.775311    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05f8c282-030f-4b3a-bbf3-58cbadb98560-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-j6xqr\" (UID: \"05f8c282-030f-4b3a-bbf3-58cbadb98560\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.777854 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.777834    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05f8c282-030f-4b3a-bbf3-58cbadb98560-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-j6xqr\" (UID: \"05f8c282-030f-4b3a-bbf3-58cbadb98560\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.786648 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.786626    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj58j\" (UniqueName: \"kubernetes.io/projected/05f8c282-030f-4b3a-bbf3-58cbadb98560-kube-api-access-gj58j\") pod \"monitoring-plugin-d5dcff7cf-j6xqr\" (UID: \"05f8c282-030f-4b3a-bbf3-58cbadb98560\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:11.891268 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:11.891224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:12.035941 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:12.035905    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"]
Jul 07 13:29:12.041509 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:12.041477    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f8c282_030f_4b3a_bbf3_58cbadb98560.slice/crio-ea01635f410f9589e4af8cdf8881b4e2540daaa1e5c06ca43bd216aa96d2b4bd WatchSource:0}: Error finding container ea01635f410f9589e4af8cdf8881b4e2540daaa1e5c06ca43bd216aa96d2b4bd: Status 404 returned error can't find the container with id ea01635f410f9589e4af8cdf8881b4e2540daaa1e5c06ca43bd216aa96d2b4bd
Jul 07 13:29:12.483215 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:12.483179    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-796f687785-gfk4v" event={"ID":"99c88269-9e09-47c2-a89e-0f87a136de78","Type":"ContainerStarted","Data":"f823a31e6247ab89281a3eca213637dc00d4c6198b40340aae12854f05b4d94f"}
Jul 07 13:29:12.485163 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:12.485130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr" event={"ID":"05f8c282-030f-4b3a-bbf3-58cbadb98560","Type":"ContainerStarted","Data":"ea01635f410f9589e4af8cdf8881b4e2540daaa1e5c06ca43bd216aa96d2b4bd"}
Jul 07 13:29:12.543975 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:12.543943    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:12.645422 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:12.645386    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:13.238610 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.238505    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:13.242550 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.241909    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:29:13.245565 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.245534    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.249190 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.249047    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 13:29:13.249553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.249375    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 13:29:13.250187 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250043    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 13:29:13.250187 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250089    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6hbcl\""
Jul 07 13:29:13.250187 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250099    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 13:29:13.250610 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250339    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 13:29:13.250610 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 13:29:13.250610 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250502    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 13:29:13.250854 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.250784    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 13:29:13.251861 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.251644    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 13:29:13.251861 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.251684    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ae5ivo13k7hec\""
Jul 07 13:29:13.252001 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.251953    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 13:29:13.252040 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.251998    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 13:29:13.256174 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.256109    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 13:29:13.261657 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.261151    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 13:29:13.262235 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.262093    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 13:29:13.284020 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.283989    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:29:13.326317 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.326284    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:13.391885 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.391836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.391903    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.391933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.391958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.391988    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392273 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-config\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392273 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392273 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392253    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392383 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392383 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392315    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392383 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392423    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-web-config\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-config-out\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8fn2\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-kube-api-access-k8fn2\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392712 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392712 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392597    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.392712 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.392624    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.433333 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.433277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:13.493268 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-config-out\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493268 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fn2\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-kube-api-access-k8fn2\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493268 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493313    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493395    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-config\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493625    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493659    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.493850 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.493800    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-web-config\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.494699 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.494282    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.496570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.494821    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.496570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.495183    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.496570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.495380    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.496570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.496270    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.496570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.496359    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.500116 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.498211    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.500116 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.498658    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-web-config\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.502241 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.501865    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.502241 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.501925    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.502241 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.501963    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-config\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.502241 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.502202    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.502983 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.502937    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.503553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.503508    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.503553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.503531    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.503711 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.503629    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.503957 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.503936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-config-out\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.505978 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.505820    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.505978 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.505926    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fn2\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-kube-api-access-k8fn2\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.506102 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.505991    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.562180 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.562143    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:13.585378 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:13.585328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:14.889499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:14.889468    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:29:14.892542 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:29:14.892512    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186b1585_e64a_49a1_86a3_0435b3a0c025.slice/crio-5a3eaf4194edd169f01035f18686b499da805413919581594ff41e7f82a4456e WatchSource:0}: Error finding container 5a3eaf4194edd169f01035f18686b499da805413919581594ff41e7f82a4456e: Status 404 returned error can't find the container with id 5a3eaf4194edd169f01035f18686b499da805413919581594ff41e7f82a4456e
Jul 07 13:29:14.998879 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:14.998791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:14.999329 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:14.999311    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7456b586d5-lk78f"]
Jul 07 13:29:15.095743 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.095681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:15.500021 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.499948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-796f687785-gfk4v" event={"ID":"99c88269-9e09-47c2-a89e-0f87a136de78","Type":"ContainerStarted","Data":"8ed26df5eb9efaa43a63cd84739f00234a76735596171c214a2a2d9ad64016e1"}
Jul 07 13:29:15.503345 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.503315    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr" event={"ID":"05f8c282-030f-4b3a-bbf3-58cbadb98560","Type":"ContainerStarted","Data":"845d90017ecb70949cffe4ab737078488bcca49722ae4b89d50533fdf98b8850"}
Jul 07 13:29:15.503505 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.503449    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:15.505490 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.505448    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"5a3eaf4194edd169f01035f18686b499da805413919581594ff41e7f82a4456e"}
Jul 07 13:29:15.507762 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.507720    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456b586d5-lk78f" event={"ID":"345e549a-20aa-4026-ab46-79bd610d2c31","Type":"ContainerStarted","Data":"222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346"}
Jul 07 13:29:15.509381 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.509361    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr"
Jul 07 13:29:15.522811 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.522739    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-796f687785-gfk4v" podStartSLOduration=1.5638098710000001 podStartE2EDuration="4.522723638s" totalImagesPullingTime="2.958913767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:15.522273009 +0000 UTC m=+90.055178727" watchObservedRunningTime="2026-07-07 13:29:15.522723638 +0000 UTC m=+90.055629364"
Jul 07 13:29:15.542759 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.542709    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7456b586d5-lk78f" podStartSLOduration=1.617151038 podStartE2EDuration="5.542694807s" totalImagesPullingTime="3.925543769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:15.54152385 +0000 UTC m=+90.074429596" watchObservedRunningTime="2026-07-07 13:29:15.542694807 +0000 UTC m=+90.075600582"
Jul 07 13:29:15.558662 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:15.558609    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-j6xqr" podStartSLOduration=1.885160731 podStartE2EDuration="4.558588214s" totalImagesPullingTime="2.673427483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:15.557856181 +0000 UTC m=+90.090761932" watchObservedRunningTime="2026-07-07 13:29:15.558588214 +0000 UTC m=+90.091493940"
Jul 07 13:29:16.513581 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:16.513542    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa" exitCode=0
Jul 07 13:29:16.514019 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:16.513656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"}
Jul 07 13:29:17.158434 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:17.158385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:17.260953 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:17.260914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:17.339863 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:17.339821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:18.001594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:18.001540    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:18.098594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:18.098557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:18.203324 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:18.203279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:18.298110 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:18.298036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:20.534194 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:20.534163    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"}
Jul 07 13:29:20.534566 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:20.534229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"}
Jul 07 13:29:20.658980 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:20.658940    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:22.511368 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:22.511336    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:22.603520 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:22.603491    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:23.549961 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.549926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"}
Jul 07 13:29:23.550432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.549992    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"}
Jul 07 13:29:23.550432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.550018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"}
Jul 07 13:29:23.550432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.550042    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerStarted","Data":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"}
Jul 07 13:29:23.563072 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.563045    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:29:23.584497 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.584444    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.853797041 podStartE2EDuration="10.584422694s" totalImagesPullingTime="6.712199147s" totalInitContainerRuntime="18.426506ms" isStatefulPod=false podCreationTimestamp="2026-07-07 13:29:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:29:23.582180781 +0000 UTC m=+98.115086534" watchObservedRunningTime="2026-07-07 13:29:23.584422694 +0000 UTC m=+98.117328420"
Jul 07 13:29:23.832069 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.831981    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:23.932936 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:23.932898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:26.349592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:26.349555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:26.440249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:26.440208    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:28.553419 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:28.553375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:29.173174 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:29.173136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:29.269603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:29.269568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:31.411550 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:31.411518    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ttjh4"
Jul 07 13:29:31.597524 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:31.597490    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:31.597693 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:31.597558    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:41.533891 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.533847    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7456b586d5-lk78f" podUID="345e549a-20aa-4026-ab46-79bd610d2c31" containerName="console" containerID="cri-o://222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346" gracePeriod=15
Jul 07 13:29:41.777398 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.777369    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:41.845992 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.845969    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-oauth-config\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846018    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-service-ca\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846064    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-oauth-serving-cert\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846080    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-serving-cert\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846102    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-trusted-ca-bundle\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846136    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-console-config\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846378 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846177    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtktv\" (UniqueName: \"kubernetes.io/projected/345e549a-20aa-4026-ab46-79bd610d2c31-kube-api-access-qtktv\") pod \"345e549a-20aa-4026-ab46-79bd610d2c31\" (UID: \"345e549a-20aa-4026-ab46-79bd610d2c31\") "
Jul 07 13:29:41.846685 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846556    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-service-ca" (OuterVolumeSpecName: "service-ca") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:29:41.846841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846661    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:29:41.846841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846692    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:29:41.846841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.846795    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-console-config" (OuterVolumeSpecName: "console-config") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:29:41.848531 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.848504    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:29:41.848683 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.848666    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:29:41.848747 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.848691    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345e549a-20aa-4026-ab46-79bd610d2c31-kube-api-access-qtktv" (OuterVolumeSpecName: "kube-api-access-qtktv") pod "345e549a-20aa-4026-ab46-79bd610d2c31" (UID: "345e549a-20aa-4026-ab46-79bd610d2c31"). InnerVolumeSpecName "kube-api-access-qtktv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:29:41.947700 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947657    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-trusted-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:41.947700 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947698    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-console-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:41.947700 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947707    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtktv\" (UniqueName: \"kubernetes.io/projected/345e549a-20aa-4026-ab46-79bd610d2c31-kube-api-access-qtktv\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:41.947700 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947716    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-oauth-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:41.948025 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947725    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-service-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:41.948025 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947734    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e549a-20aa-4026-ab46-79bd610d2c31-oauth-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:41.948025 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:41.947741    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e549a-20aa-4026-ab46-79bd610d2c31-console-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:29:42.622335 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.622300    2582 generic.go:356] "Generic (PLEG): container finished" podID="345e549a-20aa-4026-ab46-79bd610d2c31" containerID="222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346" exitCode=0
Jul 07 13:29:42.622335 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.622339    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456b586d5-lk78f"
Jul 07 13:29:42.622958 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.622353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456b586d5-lk78f" event={"ID":"345e549a-20aa-4026-ab46-79bd610d2c31","Type":"ContainerDied","Data":"222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346"}
Jul 07 13:29:42.622958 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.622411    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456b586d5-lk78f" event={"ID":"345e549a-20aa-4026-ab46-79bd610d2c31","Type":"ContainerDied","Data":"4c92985f2c34f73d8f6314a0d2eac97d80c1973455ac26f370d2321c79fdfc40"}
Jul 07 13:29:42.622958 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.622437    2582 scope.go:117] "RemoveContainer" containerID="222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346"
Jul 07 13:29:42.631181 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.631163    2582 scope.go:117] "RemoveContainer" containerID="222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346"
Jul 07 13:29:42.631430 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:29:42.631411    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346\": container with ID starting with 222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346 not found: ID does not exist" containerID="222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346"
Jul 07 13:29:42.631474 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.631439    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346"} err="failed to get container status \"222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346\": rpc error: code = NotFound desc = could not find container \"222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346\": container with ID starting with 222860584a0fb177f8a02d6f487a88019986e8ee77674c833e0c3f80e4b8c346 not found: ID does not exist"
Jul 07 13:29:42.639708 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.639675    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7456b586d5-lk78f"]
Jul 07 13:29:42.643272 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:42.643250    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7456b586d5-lk78f"]
Jul 07 13:29:44.077835 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:44.077798    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="345e549a-20aa-4026-ab46-79bd610d2c31" path="/var/lib/kubelet/pods/345e549a-20aa-4026-ab46-79bd610d2c31/volumes"
Jul 07 13:29:47.978328 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:47.978289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:48.079366 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:48.079328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:29:51.602927 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:51.602893    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:29:51.607276 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:29:51.607252    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-796f687785-gfk4v"
Jul 07 13:30:13.562841 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:13.562806    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:13.584596 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:13.584568    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:13.765953 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:13.765922    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:19.268893 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:19.268858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:30:26.083567 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:26.083527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:30:26.286315 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:26.286272    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:30:31.507065 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507031    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:30:31.507496 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507319    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="prometheus" containerID="cri-o://4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9" gracePeriod=600
Jul 07 13:30:31.507496 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507340    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy" containerID="cri-o://37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb" gracePeriod=600
Jul 07 13:30:31.507496 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507387    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="thanos-sidecar" containerID="cri-o://db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7" gracePeriod=600
Jul 07 13:30:31.507496 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507442    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-web" containerID="cri-o://4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7" gracePeriod=600
Jul 07 13:30:31.507718 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507435    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="config-reloader" containerID="cri-o://e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b" gracePeriod=600
Jul 07 13:30:31.507718 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.507450    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e" gracePeriod=600
Jul 07 13:30:31.762726 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.762667    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:31.821131 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821097    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e" exitCode=0
Jul 07 13:30:31.821131 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821128    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb" exitCode=0
Jul 07 13:30:31.821131 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821138    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7" exitCode=0
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821148    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7" exitCode=0
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821146    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"}
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821190    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"}
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821156    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b" exitCode=0
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821244    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"}
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821245    2582 generic.go:356] "Generic (PLEG): container finished" podID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9" exitCode=0
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821271    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"}
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"}
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821321    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"}
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821319    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.821427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.821347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186b1585-e64a-49a1-86a3-0435b3a0c025","Type":"ContainerDied","Data":"5a3eaf4194edd169f01035f18686b499da805413919581594ff41e7f82a4456e"}
Jul 07 13:30:31.833140 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.831966    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.843910 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.843886    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.852090 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.852067    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.858515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858481    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.858641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858520    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-serving-certs-ca-bundle\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.858641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858542    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-grpc-tls\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.858641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858578    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-config-out\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.858913 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858886    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-tls\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.858913 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858904    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.859097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858933    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.859097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-2\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.859097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.858992    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-metrics-client-certs\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859511    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-db\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859556    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-web-config\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859592    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-kubelet-serving-ca-bundle\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859597    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859631    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-kube-rbac-proxy\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859670    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-config\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859694    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8fn2\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-kube-api-access-k8fn2\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859748    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-metrics-client-ca\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859807    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-trusted-ca-bundle\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859833    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-tls-assets\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859863    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-0\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859900    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-1\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.859942    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-thanos-prometheus-http-client-file\") pod \"186b1585-e64a-49a1-86a3-0435b3a0c025\" (UID: \"186b1585-e64a-49a1-86a3-0435b3a0c025\") "
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.860222    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.860241    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.860367    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.860826 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.860793    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.861924 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.861149    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.861924 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.861212    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.862525 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.862366    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.862525 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.862457    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:30:31.862954 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.862749    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:30:31.862954 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.862792    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-config-out" (OuterVolumeSpecName: "config-out") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:30:31.864272 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.864231    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.864588 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.864519    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.864851 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.864813    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.864938 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.864865    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.865003 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.864942    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.865058 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.865003    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.865170 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.865151    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.865466 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.865437    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:30:31.865840 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.865817    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-config" (OuterVolumeSpecName: "config") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.866603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.866579    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-kube-api-access-k8fn2" (OuterVolumeSpecName: "kube-api-access-k8fn2") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "kube-api-access-k8fn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:30:31.874265 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.874159    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.875898 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.875874    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-web-config" (OuterVolumeSpecName: "web-config") pod "186b1585-e64a-49a1-86a3-0435b3a0c025" (UID: "186b1585-e64a-49a1-86a3-0435b3a0c025"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:30:31.882338 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.882321    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.888832 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.888753    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.889052 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.889034    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.889097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889063    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"} err="failed to get container status \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist"
Jul 07 13:30:31.889097 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889082    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.889312 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.889296    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.889366 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889318    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"} err="failed to get container status \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist"
Jul 07 13:30:31.889366 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889339    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.889611 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.889585    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.889680 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889619    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"} err="failed to get container status \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist"
Jul 07 13:30:31.889680 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889634    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.889912 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.889894    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.889964 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889918    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"} err="failed to get container status \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist"
Jul 07 13:30:31.889964 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.889932    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.890175 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.890159    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.890286 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890178    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"} err="failed to get container status \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist"
Jul 07 13:30:31.890286 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890191    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.890432 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.890416    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.890467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890437    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"} err="failed to get container status \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist"
Jul 07 13:30:31.890467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890451    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.890685 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:30:31.890670    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.890726 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890689    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"} err="failed to get container status \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist"
Jul 07 13:30:31.890726 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890702    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.890949 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890931    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"} err="failed to get container status \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist"
Jul 07 13:30:31.890949 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.890948    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.891152 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891137    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"} err="failed to get container status \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist"
Jul 07 13:30:31.891206 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891153    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.891358 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891341    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"} err="failed to get container status \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist"
Jul 07 13:30:31.891399 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891362    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.891602 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891583    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"} err="failed to get container status \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist"
Jul 07 13:30:31.891602 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891601    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.891842 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891825    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"} err="failed to get container status \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist"
Jul 07 13:30:31.891842 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.891842    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.894742 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.892359    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"} err="failed to get container status \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist"
Jul 07 13:30:31.894742 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.892399    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.896982 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.896957    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"} err="failed to get container status \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist"
Jul 07 13:30:31.896982 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.896981    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.897258 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897237    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"} err="failed to get container status \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist"
Jul 07 13:30:31.897258 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897258    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.897525 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897504    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"} err="failed to get container status \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist"
Jul 07 13:30:31.897579 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897526    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.897730 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897716    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"} err="failed to get container status \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist"
Jul 07 13:30:31.897796 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897732    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.897957 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897934    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"} err="failed to get container status \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist"
Jul 07 13:30:31.898008 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.897960    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.898221 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898193    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"} err="failed to get container status \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist"
Jul 07 13:30:31.898221 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898213    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.898436 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898419    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"} err="failed to get container status \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist"
Jul 07 13:30:31.898488 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898437    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.898640 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898624    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"} err="failed to get container status \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist"
Jul 07 13:30:31.898682 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898640    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.898843 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898828    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"} err="failed to get container status \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist"
Jul 07 13:30:31.898890 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.898844    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.899058 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899042    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"} err="failed to get container status \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist"
Jul 07 13:30:31.899058 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899057    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.899241 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899222    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"} err="failed to get container status \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist"
Jul 07 13:30:31.899281 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899243    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.899406 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899389    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"} err="failed to get container status \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist"
Jul 07 13:30:31.899406 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899407    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.899591 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899577    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"} err="failed to get container status \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist"
Jul 07 13:30:31.899630 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899591    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.899808 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899788    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"} err="failed to get container status \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist"
Jul 07 13:30:31.899808 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899808    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.900004 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.899989    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"} err="failed to get container status \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist"
Jul 07 13:30:31.900039 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900004    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.900186 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900170    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"} err="failed to get container status \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist"
Jul 07 13:30:31.900226 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900186    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.900363 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900348    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"} err="failed to get container status \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist"
Jul 07 13:30:31.900400 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900363    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.900563 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900545    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"} err="failed to get container status \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist"
Jul 07 13:30:31.900603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900564    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.900790 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900756    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"} err="failed to get container status \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist"
Jul 07 13:30:31.900790 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.900786    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.901027 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901007    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"} err="failed to get container status \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist"
Jul 07 13:30:31.901027 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901023    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.901231 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901215    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"} err="failed to get container status \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist"
Jul 07 13:30:31.901278 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901232    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.901420 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901403    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"} err="failed to get container status \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist"
Jul 07 13:30:31.901461 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901421    2582 scope.go:117] "RemoveContainer" containerID="3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"
Jul 07 13:30:31.901629 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901610    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e"} err="failed to get container status \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": rpc error: code = NotFound desc = could not find container \"3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e\": container with ID starting with 3d8fff1833bb1a3537dcbd8e097964688a0c57cc443f9c9c2b81297f408cd48e not found: ID does not exist"
Jul 07 13:30:31.901720 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901632    2582 scope.go:117] "RemoveContainer" containerID="37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"
Jul 07 13:30:31.901859 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901844    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb"} err="failed to get container status \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": rpc error: code = NotFound desc = could not find container \"37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb\": container with ID starting with 37c47bda981393fac2686273e9066affe8ae3e3f8b6c07e016d217dd0339c9bb not found: ID does not exist"
Jul 07 13:30:31.901912 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.901860    2582 scope.go:117] "RemoveContainer" containerID="4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"
Jul 07 13:30:31.902026 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902005    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7"} err="failed to get container status \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": rpc error: code = NotFound desc = could not find container \"4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7\": container with ID starting with 4334798ef46b6bfb268fec700d64b74c45f17c98121d5ddd9e46589a421b62b7 not found: ID does not exist"
Jul 07 13:30:31.902080 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902027    2582 scope.go:117] "RemoveContainer" containerID="db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"
Jul 07 13:30:31.902270 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902252    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7"} err="failed to get container status \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": rpc error: code = NotFound desc = could not find container \"db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7\": container with ID starting with db285dee353881177edbce13ffb37c892a1cc1427eb26b9fc4d3804d868985b7 not found: ID does not exist"
Jul 07 13:30:31.902326 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902270    2582 scope.go:117] "RemoveContainer" containerID="e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"
Jul 07 13:30:31.902463 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902443    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b"} err="failed to get container status \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": rpc error: code = NotFound desc = could not find container \"e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b\": container with ID starting with e7491e081805fc7dfb4650298c71f17a8049152af2893a5ec24acd4029a9bd2b not found: ID does not exist"
Jul 07 13:30:31.902513 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902463    2582 scope.go:117] "RemoveContainer" containerID="4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"
Jul 07 13:30:31.902640 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902623    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9"} err="failed to get container status \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": rpc error: code = NotFound desc = could not find container \"4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9\": container with ID starting with 4fe4041936e67205d05d277e15dbcf939099cbf7268e7d8bff4aeea6cb37cbe9 not found: ID does not exist"
Jul 07 13:30:31.902682 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902641    2582 scope.go:117] "RemoveContainer" containerID="acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"
Jul 07 13:30:31.902877 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.902862    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa"} err="failed to get container status \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": rpc error: code = NotFound desc = could not find container \"acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa\": container with ID starting with acf2d8785eeffcf355b8bccab54acb75e8c6698be86a6d6a37fe3a91310139fa not found: ID does not exist"
Jul 07 13:30:31.961470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961436    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961465    2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961477    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961486    2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-grpc-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961496    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-config-out\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961504    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961513    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961522    2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-metrics-client-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961530    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-db\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961560    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-web-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961569    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961578    2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-secret-kube-rbac-proxy\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961587    2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/186b1585-e64a-49a1-86a3-0435b3a0c025-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961596    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8fn2\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-kube-api-access-k8fn2\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961605    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-configmap-metrics-client-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961613    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961621    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186b1585-e64a-49a1-86a3-0435b3a0c025-tls-assets\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:31.961698 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:31.961629    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186b1585-e64a-49a1-86a3-0435b3a0c025-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:30:32.142924 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.142887    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:30:32.148426 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.148392    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:30:32.179668 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.179631    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:30:32.179980 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.179963    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-thanos"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.179983    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-thanos"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.179998    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="prometheus"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180006    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="prometheus"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180018    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180026    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180041    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="thanos-sidecar"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180049    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="thanos-sidecar"
Jul 07 13:30:32.180070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180067    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="init-config-reloader"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180075    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="init-config-reloader"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180084    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-web"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180093    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-web"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180104    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="config-reloader"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180113    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="config-reloader"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180122    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345e549a-20aa-4026-ab46-79bd610d2c31" containerName="console"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180131    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="345e549a-20aa-4026-ab46-79bd610d2c31" containerName="console"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180203    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="prometheus"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180213    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="config-reloader"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180222    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-web"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180231    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180239    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="thanos-sidecar"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180250    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="345e549a-20aa-4026-ab46-79bd610d2c31" containerName="console"
Jul 07 13:30:32.180444 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.180259    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" containerName="kube-rbac-proxy-thanos"
Jul 07 13:30:32.184734 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.184712    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.187705 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.187686    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 13:30:32.187882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.187830    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 13:30:32.187882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.187841    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 13:30:32.187882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.187867    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6hbcl\""
Jul 07 13:30:32.188115 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.187908    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ae5ivo13k7hec\""
Jul 07 13:30:32.188115 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188045    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 13:30:32.188115 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188072    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 13:30:32.188302 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188124    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 13:30:32.188302 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188149    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 13:30:32.188302 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188181    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 13:30:32.188430 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188386    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 13:30:32.188430 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188407    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 13:30:32.188580 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188556    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 13:30:32.188810 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.188796    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 13:30:32.193346 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.193329    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 13:30:32.195033 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.195016    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 13:30:32.206694 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.206672    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:30:32.264460 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264460 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn5p4\" (UniqueName: \"kubernetes.io/projected/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-kube-api-access-fn5p4\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264555    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-config\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264581    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264619    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264636    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264689    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264709    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264761    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.264844 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.265149 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264851    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.265149 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264871    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.265149 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.264912    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.365671 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-config\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.365671 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.365904 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.365904 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.365904 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.365904 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365900    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.365986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn5p4\" (UniqueName: \"kubernetes.io/projected/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-kube-api-access-fn5p4\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366509    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.366946 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366885    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.367004 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.366948    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.367795 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.367474    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.368812 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.368788    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-config\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.368812 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.368799    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.368997 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.368889    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.369232 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369124    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369487    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369626    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369706    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369814    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369889    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370073 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.369901    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.370353 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.370238    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.371243 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.371213    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.371354 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.371334    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.371720 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.371702    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.372620 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.372599    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.378806 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.378786    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn5p4\" (UniqueName: \"kubernetes.io/projected/9e2d393f-84f8-4ad0-a809-dc7ae03823c3-kube-api-access-fn5p4\") pod \"prometheus-k8s-0\" (UID: \"9e2d393f-84f8-4ad0-a809-dc7ae03823c3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.495128 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.495078    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:32.637688 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.637615    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 13:30:32.640640 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:30:32.640611    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2d393f_84f8_4ad0_a809_dc7ae03823c3.slice/crio-6d00d9675183ecad25f92b9394942efc76dc8e78178424ab4d4637eb1c47cfab WatchSource:0}: Error finding container 6d00d9675183ecad25f92b9394942efc76dc8e78178424ab4d4637eb1c47cfab: Status 404 returned error can't find the container with id 6d00d9675183ecad25f92b9394942efc76dc8e78178424ab4d4637eb1c47cfab
Jul 07 13:30:32.827201 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.827167    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e2d393f-84f8-4ad0-a809-dc7ae03823c3" containerID="f261e68857c60e7dd839ae1fe26a2f7cd06e6cb6bdc0985ae23fb2a4830fa5de" exitCode=0
Jul 07 13:30:32.827392 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.827229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerDied","Data":"f261e68857c60e7dd839ae1fe26a2f7cd06e6cb6bdc0985ae23fb2a4830fa5de"}
Jul 07 13:30:32.827392 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:32.827289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"6d00d9675183ecad25f92b9394942efc76dc8e78178424ab4d4637eb1c47cfab"}
Jul 07 13:30:33.833983 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.833892    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"8a6f9cd3114cca9139376c450b6c4e778399f4edffc4a90c226756d3f5b57cbb"}
Jul 07 13:30:33.834438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.834004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"ca42e8e222591631f74a3392611f1e7c8888f82fc974f91a8a4fb770672a69fc"}
Jul 07 13:30:33.834438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.834053    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"839344e51f42727cee0054733f81efd737c3e65c4736ad9b0d0afcf9c529a00d"}
Jul 07 13:30:33.834438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.834099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"4bc05947fd53c689ced9576f846bb9e257c6c95dd4dd478e0d8b93fc75e82ae6"}
Jul 07 13:30:33.834438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.834144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"30f8b4a52bdd8605800ae1710a0ba7520075642e9dc221437d4a10a118b82b41"}
Jul 07 13:30:33.834438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.834185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e2d393f-84f8-4ad0-a809-dc7ae03823c3","Type":"ContainerStarted","Data":"933d84cd544be7dc7aef260e9a3f8a23e09f3e6e978e539196838ea6fac3bced"}
Jul 07 13:30:33.867685 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:33.867633    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.8487029339999999 podStartE2EDuration="1.86761569s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.912756ms" isStatefulPod=false podCreationTimestamp="2026-07-07 13:30:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:30:33.865653144 +0000 UTC m=+168.398558888" watchObservedRunningTime="2026-07-07 13:30:33.86761569 +0000 UTC m=+168.400521416"
Jul 07 13:30:34.078089 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:34.078057    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186b1585-e64a-49a1-86a3-0435b3a0c025" path="/var/lib/kubelet/pods/186b1585-e64a-49a1-86a3-0435b3a0c025/volumes"
Jul 07 13:30:35.755487 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:35.755447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:30:35.847110 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:35.847047    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:30:37.496147 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:37.496055    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:30:46.989218 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:46.989170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:30:47.120161 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:30:47.120125    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:18.145830 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:18.145790    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:22.291644 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:22.291606    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:22.378958 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:22.378921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:32.495284 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:32.495244    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:31:32.510679 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:32.510654    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:31:33.071828 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:33.071802    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 13:31:41.393125 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:41.393086    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:41.493886 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:41.493855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:41.786572 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:41.786488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:41.885188 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:41.885151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:41.980828 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:41.980793    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:42.538736 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:42.538701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:42.640405 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:42.640369    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:51.958539 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:51.958502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:52.229105 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:52.228993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:52.359972 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:52.359938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:52.431605 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:52.431571    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:53.588763 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:53.588708    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:31:53.706250 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:31:53.706219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:02.193751 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:02.193686    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:02.280915 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:02.280881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:02.354547 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:02.354513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:10.281769 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:10.281683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:41.373131 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:41.373091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:43.385830 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:43.385756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:32:45.915343 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:32:45.915317    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 13:33:08.660886 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:08.660851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:09.063209 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:09.063115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:12.705808 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:12.705757    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:12.802017 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:12.801978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:13.844221 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:13.844193    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:13.939171 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:13.939142    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:14.716540 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:14.716506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:14.813133 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:14.813065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:16.854307 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:16.854271    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:17.056120 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:17.056074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:19.272466 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:19.272437    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:19.347336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:19.347302    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:25.026363 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:25.026333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:26.550042 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:26.550010    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:26.650457 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:26.650427    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:32.709623 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:32.709584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:32.790171 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:32.790133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:34.328083 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:34.328047    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:46.510878 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:46.510801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:46.627695 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:46.627655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:47.155043 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.155007    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:47.573353 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.573273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:47.668665 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.668635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:47.735959 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.735861    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"]
Jul 07 13:33:47.738223 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.738207    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.743425 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.743390    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 13:33:47.743580 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.743561    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 13:33:47.743641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.743581    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nb9q7\""
Jul 07 13:33:47.743641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.743606    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 13:33:47.743945 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.743928    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 13:33:47.797185 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.797158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"]
Jul 07 13:33:47.866104 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.866071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.866248 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.866120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.866248 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.866186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-manifests\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.866248 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.866207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-tmp\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.866355 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.866255    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66vdm\" (UniqueName: \"kubernetes.io/projected/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-kube-api-access-66vdm\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.907432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.907397    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:47.968383 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.968358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-manifests\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.968557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.968392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-tmp\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.968557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.968414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66vdm\" (UniqueName: \"kubernetes.io/projected/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-kube-api-access-66vdm\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.968557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.968446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.968557 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.968482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.968847 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.968752    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-manifests\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.970574 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.970546    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-tmp\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.970999 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.970980    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.971071 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.970980    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:47.979115 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:47.979093    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66vdm\" (UniqueName: \"kubernetes.io/projected/b6295a72-6bd0-401e-a6f6-5790bf4c3fdc-kube-api-access-66vdm\") pod \"opendatahub-operator-controller-manager-76f9489749-767tj\" (UID: \"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc\") " pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:48.007920 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.007855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:48.047403 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.047376    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:48.109465 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.109290    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:48.183569 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.183492    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"]
Jul 07 13:33:48.188112 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:33:48.188082    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6295a72_6bd0_401e_a6f6_5790bf4c3fdc.slice/crio-9101f322cfa0c282864a3048e1eda614f9098482ea2130a66892ce5d05234c7c WatchSource:0}: Error finding container 9101f322cfa0c282864a3048e1eda614f9098482ea2130a66892ce5d05234c7c: Status 404 returned error can't find the container with id 9101f322cfa0c282864a3048e1eda614f9098482ea2130a66892ce5d05234c7c
Jul 07 13:33:48.189753 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.189732    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:33:48.227744 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.227714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:48.354405 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.354372    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:48.455220 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.455127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:48.547968 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.547938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj" event={"ID":"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc","Type":"ContainerStarted","Data":"9101f322cfa0c282864a3048e1eda614f9098482ea2130a66892ce5d05234c7c"}
Jul 07 13:33:48.551994 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:48.551964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:51.561992 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:51.561904    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj" event={"ID":"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc","Type":"ContainerStarted","Data":"a0b03089e6f00e51d4106fac863e2f70e93a085f773919a7dfa9b105e16a94ae"}
Jul 07 13:33:52.566716 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:52.566683    2582 generic.go:356] "Generic (PLEG): container finished" podID="b6295a72-6bd0-401e-a6f6-5790bf4c3fdc" containerID="a0b03089e6f00e51d4106fac863e2f70e93a085f773919a7dfa9b105e16a94ae" exitCode=0
Jul 07 13:33:52.567152 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:52.566768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj" event={"ID":"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc","Type":"ContainerDied","Data":"a0b03089e6f00e51d4106fac863e2f70e93a085f773919a7dfa9b105e16a94ae"}
Jul 07 13:33:53.572176 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:53.572096    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj" event={"ID":"b6295a72-6bd0-401e-a6f6-5790bf4c3fdc","Type":"ContainerStarted","Data":"9e6c508f6bcb728cd6db3cdefac679adbbb9a3369edfe2754a9d0df4b1fe7289"}
Jul 07 13:33:53.572511 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:53.572213    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:33:53.607050 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:53.607008    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj" podStartSLOduration=1.791443165 podStartE2EDuration="6.6069917s" totalImagesPullingTime="3.717715163s" totalInitContainerRuntime="1.097833372s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:33:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:33:53.606757708 +0000 UTC m=+368.139663433" watchObservedRunningTime="2026-07-07 13:33:53.6069917 +0000 UTC m=+368.139897425"
Jul 07 13:33:54.753618 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:54.753585    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:33:54.817487 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:33:54.817456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:04.578407 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:04.578380    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76f9489749-767tj"
Jul 07 13:34:06.715790 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:06.715749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:06.846914 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:06.846882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:07.515969 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.515935    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"]
Jul 07 13:34:07.525892 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.525867    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.529482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.529458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 13:34:07.529482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.529476    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 13:34:07.529766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.529466    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 13:34:07.529766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.529562    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 13:34:07.529766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.529592    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-db7gh\""
Jul 07 13:34:07.529766 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.529644    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:34:07.533713 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.533692    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"]
Jul 07 13:34:07.565589 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.565537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:07.639677 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.639649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f7146892-c42b-4032-95ff-44e174195312-manager-config\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.639677 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.639685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7146892-c42b-4032-95ff-44e174195312-metrics-cert\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.639904 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.639717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7146892-c42b-4032-95ff-44e174195312-cert\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.639904 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.639757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s672q\" (UniqueName: \"kubernetes.io/projected/f7146892-c42b-4032-95ff-44e174195312-kube-api-access-s672q\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.662270 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.662239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:07.740854 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.740817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f7146892-c42b-4032-95ff-44e174195312-manager-config\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.741316 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.740869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7146892-c42b-4032-95ff-44e174195312-metrics-cert\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.741316 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.740901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7146892-c42b-4032-95ff-44e174195312-cert\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.741316 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.740927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s672q\" (UniqueName: \"kubernetes.io/projected/f7146892-c42b-4032-95ff-44e174195312-kube-api-access-s672q\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.741603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.741577    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f7146892-c42b-4032-95ff-44e174195312-manager-config\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.743429 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.743399    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7146892-c42b-4032-95ff-44e174195312-metrics-cert\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.743517 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.743453    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7146892-c42b-4032-95ff-44e174195312-cert\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.758543 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.758512    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s672q\" (UniqueName: \"kubernetes.io/projected/f7146892-c42b-4032-95ff-44e174195312-kube-api-access-s672q\") pod \"lws-controller-manager-5dbf57879c-2gfcs\" (UID: \"f7146892-c42b-4032-95ff-44e174195312\") " pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.760974 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.760950    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:07.836211 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.836125    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:07.868233 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.868204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:07.969925 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:07.969888    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"]
Jul 07 13:34:07.972680 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:34:07.972652    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7146892_c42b_4032_95ff_44e174195312.slice/crio-66f158023cf4762e8b94da236d66940cb4a4de9b6410a312896ea3ea4eaece66 WatchSource:0}: Error finding container 66f158023cf4762e8b94da236d66940cb4a4de9b6410a312896ea3ea4eaece66: Status 404 returned error can't find the container with id 66f158023cf4762e8b94da236d66940cb4a4de9b6410a312896ea3ea4eaece66
Jul 07 13:34:08.309356 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:08.309324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:08.571631 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:08.571515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:08.635873 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:08.635841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs" event={"ID":"f7146892-c42b-4032-95ff-44e174195312","Type":"ContainerStarted","Data":"66f158023cf4762e8b94da236d66940cb4a4de9b6410a312896ea3ea4eaece66"}
Jul 07 13:34:10.645446 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:10.645414    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs" event={"ID":"f7146892-c42b-4032-95ff-44e174195312","Type":"ContainerStarted","Data":"3ad1af97e1616a4d8ba9723a0a12898c272ce26ee0cae49ceebebe64d66045ed"}
Jul 07 13:34:10.645836 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:10.645516    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:10.666292 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:10.666239    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs" podStartSLOduration=1.172845995 podStartE2EDuration="3.666220387s" totalImagesPullingTime="2.493374392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:34:10.665053057 +0000 UTC m=+385.197958783" watchObservedRunningTime="2026-07-07 13:34:10.666220387 +0000 UTC m=+385.199126115"
Jul 07 13:34:12.876423 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.876384    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:12.912357 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.912329    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"]
Jul 07 13:34:12.914561 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.914540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:12.919291 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.919272    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 13:34:12.922328 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.922310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m875t\""
Jul 07 13:34:12.922684 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.922664    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 13:34:12.923500 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.923484    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 13:34:12.924706 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.924692    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 13:34:12.935953 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.935933    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"]
Jul 07 13:34:12.988352 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:12.988314    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:13.089637 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.089610    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6kl8\" (UniqueName: \"kubernetes.io/projected/af2337ea-212c-4499-9d55-35f9a0b95e3d-kube-api-access-b6kl8\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.089783 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.089662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af2337ea-212c-4499-9d55-35f9a0b95e3d-tmp\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.089783 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.089755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af2337ea-212c-4499-9d55-35f9a0b95e3d-tls-certs\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.191249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.191162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af2337ea-212c-4499-9d55-35f9a0b95e3d-tls-certs\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.191249 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.191224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6kl8\" (UniqueName: \"kubernetes.io/projected/af2337ea-212c-4499-9d55-35f9a0b95e3d-kube-api-access-b6kl8\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.191467 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.191296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af2337ea-212c-4499-9d55-35f9a0b95e3d-tmp\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.193465 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.193444    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af2337ea-212c-4499-9d55-35f9a0b95e3d-tmp\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.193652 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.193634    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af2337ea-212c-4499-9d55-35f9a0b95e3d-tls-certs\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.203482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.203463    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6kl8\" (UniqueName: \"kubernetes.io/projected/af2337ea-212c-4499-9d55-35f9a0b95e3d-kube-api-access-b6kl8\") pod \"kube-auth-proxy-87699d5c5-zxqbx\" (UID: \"af2337ea-212c-4499-9d55-35f9a0b95e3d\") " pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.224502 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.224482    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"
Jul 07 13:34:13.353741 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.353597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx"]
Jul 07 13:34:13.356223 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:34:13.356197    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf2337ea_212c_4499_9d55_35f9a0b95e3d.slice/crio-0d06d56fa1b65ca94dc4298e94b47a29ff2a5509f4bf5f6a61a82ab9454f27fd WatchSource:0}: Error finding container 0d06d56fa1b65ca94dc4298e94b47a29ff2a5509f4bf5f6a61a82ab9454f27fd: Status 404 returned error can't find the container with id 0d06d56fa1b65ca94dc4298e94b47a29ff2a5509f4bf5f6a61a82ab9454f27fd
Jul 07 13:34:13.658922 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:13.658887    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx" event={"ID":"af2337ea-212c-4499-9d55-35f9a0b95e3d","Type":"ContainerStarted","Data":"0d06d56fa1b65ca94dc4298e94b47a29ff2a5509f4bf5f6a61a82ab9454f27fd"}
Jul 07 13:34:14.911183 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:14.911130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:15.012603 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:15.012566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:16.610007 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:16.605888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:16.674566 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:16.674488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx" event={"ID":"af2337ea-212c-4499-9d55-35f9a0b95e3d","Type":"ContainerStarted","Data":"e1d60c294b82e36fcfe2303f0f300949cbf6fe978fb7bddb2facc771a6a48b86"}
Jul 07 13:34:16.699662 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:16.699596    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-87699d5c5-zxqbx" podStartSLOduration=1.638644759 podStartE2EDuration="4.699582494s" totalImagesPullingTime="3.060937735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:34:16.69774339 +0000 UTC m=+391.230649115" watchObservedRunningTime="2026-07-07 13:34:16.699582494 +0000 UTC m=+391.232488215"
Jul 07 13:34:19.538980 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:19.538937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:19.675165 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:19.675123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:19.824113 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:19.824019    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:19.925101 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:19.925071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:20.178575 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:20.178540    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:20.283398 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:20.283356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:20.995594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:20.995558    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:21.116757 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:21.116725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:21.247081 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:21.246993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:21.651594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:21.651564    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5dbf57879c-2gfcs"
Jul 07 13:34:21.990233 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:21.990134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:22.239007 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:22.238970    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:22.333239 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:22.333140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:36.536301 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:36.536269    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:40.512980 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:40.512939    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:34:40.584757 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:34:40.584720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:08.196909 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:08.196834    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:08.391879 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:08.391842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:08.499326 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:08.499239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:09.085617 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:09.085580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:09.290230 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:09.290175    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:10.057506 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:10.057473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:10.170197 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:10.170167    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:38.391726 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:38.391690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:41.770715 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:41.770667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:44.862796 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:44.862751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:46.239109 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:46.239069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:47.159976 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:47.159938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:49.333338 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:49.333303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:49.722609 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:49.722572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:50.123515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:50.123486    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:50.256386 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:50.256353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:52.193541 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.193506    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"]
Jul 07 13:35:52.195947 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.195929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:35:52.199978 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.199951    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 13:35:52.201044 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.201022    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 13:35:52.201153 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.201022    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wp69z\""
Jul 07 13:35:52.210622 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.210593    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"]
Jul 07 13:35:52.342594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.342559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwh7n\" (UniqueName: \"kubernetes.io/projected/744040c2-d02e-4212-acb6-48e0d59bfe0c-kube-api-access-nwh7n\") pod \"authorino-operator-75cc8d5cb7-b6ntf\" (UID: \"744040c2-d02e-4212-acb6-48e0d59bfe0c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:35:52.443199 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.443166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwh7n\" (UniqueName: \"kubernetes.io/projected/744040c2-d02e-4212-acb6-48e0d59bfe0c-kube-api-access-nwh7n\") pod \"authorino-operator-75cc8d5cb7-b6ntf\" (UID: \"744040c2-d02e-4212-acb6-48e0d59bfe0c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:35:52.457600 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.457534    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwh7n\" (UniqueName: \"kubernetes.io/projected/744040c2-d02e-4212-acb6-48e0d59bfe0c-kube-api-access-nwh7n\") pod \"authorino-operator-75cc8d5cb7-b6ntf\" (UID: \"744040c2-d02e-4212-acb6-48e0d59bfe0c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:35:52.506843 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.506811    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:35:52.636490 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.636356    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"]
Jul 07 13:35:52.639256 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:35:52.639229    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744040c2_d02e_4212_acb6_48e0d59bfe0c.slice/crio-d696cb7cc644e49b3bafa076cb9f27da42994616d0be30937813df4bbec89e35 WatchSource:0}: Error finding container d696cb7cc644e49b3bafa076cb9f27da42994616d0be30937813df4bbec89e35: Status 404 returned error can't find the container with id d696cb7cc644e49b3bafa076cb9f27da42994616d0be30937813df4bbec89e35
Jul 07 13:35:52.771477 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:52.771400    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:53.075278 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:53.075196    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf" event={"ID":"744040c2-d02e-4212-acb6-48e0d59bfe0c","Type":"ContainerStarted","Data":"d696cb7cc644e49b3bafa076cb9f27da42994616d0be30937813df4bbec89e35"}
Jul 07 13:35:54.163553 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:54.163519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:54.266209 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:54.266178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:55.466422 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:55.466385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:55.577432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:55.577368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:57.096204 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:57.096174    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf" event={"ID":"744040c2-d02e-4212-acb6-48e0d59bfe0c","Type":"ContainerStarted","Data":"afb5928c915660a893b7394dc0fe032fa5a40d50e4569581387822a54ec49c11"}
Jul 07 13:35:57.096576 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:57.096299    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:35:57.114354 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:57.114304    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf" podStartSLOduration=1.281558457 podStartE2EDuration="5.11428753s" totalImagesPullingTime="3.832729073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:35:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:35:57.114285362 +0000 UTC m=+491.647191089" watchObservedRunningTime="2026-07-07 13:35:57.11428753 +0000 UTC m=+491.647193258"
Jul 07 13:35:57.963512 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:57.963477    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:35:58.052606 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:35:58.052566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:05.497617 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:05.497583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:05.619734 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:05.619697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:08.102816 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:08.102763    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b6ntf"
Jul 07 13:36:10.170225 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:10.170189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:10.271924 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:10.271888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:12.766730 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:12.766688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:12.896411 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:12.896374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:16.171451 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:16.171407    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:16.281000 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:16.280958    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:24.741737 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:24.741701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:24.868764 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:24.868728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:30.521087 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:30.521052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:30.598503 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:30.598468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:31.137788 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:31.137751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:31.236927 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:31.236886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:37.513956 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:37.513869    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:37.617961 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:37.617919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:40.932204 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:40.932165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:41.238403 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:41.238315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:42.565726 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:42.565681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:42.668690 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:42.668651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:43.099184 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.099157    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:43.555050 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.554970    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:36:43.557263 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.557249    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.560248 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.560229    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qcjrm\""
Jul 07 13:36:43.563442 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.563419    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 13:36:43.579711 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.579689    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:36:43.602364 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.602338    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:36:43.674499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.674467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65c1a5ef-0724-425c-9728-a660a386ad43-config-file\") pod \"limitador-limitador-54d6d988d5-r4jvr\" (UID: \"65c1a5ef-0724-425c-9728-a660a386ad43\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.674667 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.674522    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57plj\" (UniqueName: \"kubernetes.io/projected/65c1a5ef-0724-425c-9728-a660a386ad43-kube-api-access-57plj\") pod \"limitador-limitador-54d6d988d5-r4jvr\" (UID: \"65c1a5ef-0724-425c-9728-a660a386ad43\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.775839 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.775749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57plj\" (UniqueName: \"kubernetes.io/projected/65c1a5ef-0724-425c-9728-a660a386ad43-kube-api-access-57plj\") pod \"limitador-limitador-54d6d988d5-r4jvr\" (UID: \"65c1a5ef-0724-425c-9728-a660a386ad43\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.776041 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.775949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65c1a5ef-0724-425c-9728-a660a386ad43-config-file\") pod \"limitador-limitador-54d6d988d5-r4jvr\" (UID: \"65c1a5ef-0724-425c-9728-a660a386ad43\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.776565 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.776545    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65c1a5ef-0724-425c-9728-a660a386ad43-config-file\") pod \"limitador-limitador-54d6d988d5-r4jvr\" (UID: \"65c1a5ef-0724-425c-9728-a660a386ad43\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.786232 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.786204    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57plj\" (UniqueName: \"kubernetes.io/projected/65c1a5ef-0724-425c-9728-a660a386ad43-kube-api-access-57plj\") pod \"limitador-limitador-54d6d988d5-r4jvr\" (UID: \"65c1a5ef-0724-425c-9728-a660a386ad43\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:43.867358 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:43.867326    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:44.005602 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:44.005574    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:36:44.009649 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:36:44.009601    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c1a5ef_0724_425c_9728_a660a386ad43.slice/crio-f19d28ee4b8673e0293e359c08a7785746e7f3a30641f22aadf1403093a1ae19 WatchSource:0}: Error finding container f19d28ee4b8673e0293e359c08a7785746e7f3a30641f22aadf1403093a1ae19: Status 404 returned error can't find the container with id f19d28ee4b8673e0293e359c08a7785746e7f3a30641f22aadf1403093a1ae19
Jul 07 13:36:44.296726 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:44.296647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr" event={"ID":"65c1a5ef-0724-425c-9728-a660a386ad43","Type":"ContainerStarted","Data":"f19d28ee4b8673e0293e359c08a7785746e7f3a30641f22aadf1403093a1ae19"}
Jul 07 13:36:44.482677 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:44.482639    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:44.582067 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:44.581979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:48.314745 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:48.314659    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr" event={"ID":"65c1a5ef-0724-425c-9728-a660a386ad43","Type":"ContainerStarted","Data":"aa86ee01749c0a111dd69eaace0c5d1c807c22e4b0a911be48b43e4e8ef299c4"}
Jul 07 13:36:48.314745 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:48.314742    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:36:48.341170 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:48.341121    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr" podStartSLOduration=1.4481726240000001 podStartE2EDuration="5.341103616s" totalImagesPullingTime="3.892930992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:36:48.339522854 +0000 UTC m=+542.872428605" watchObservedRunningTime="2026-07-07 13:36:48.341103616 +0000 UTC m=+542.874009341"
Jul 07 13:36:48.398700 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:48.398667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:49.770613 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:49.770581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:49.874882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:49.874821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:50.456892 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:50.456862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:50.557002 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:50.556965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:54.503785 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:54.503715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:54.610061 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:54.610022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:54.913573 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:54.913521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:54.997979 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:54.997945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:56.352531 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:56.352499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:36:59.319823 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:36:59.319792    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"
Jul 07 13:37:05.977068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:05.977036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:17.403717 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:17.403673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:17.579229 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:17.579195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:17.994475 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:17.994435    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-974d64977-n4fcx"]
Jul 07 13:37:18.002632 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.002604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.005757 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.005733    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 13:37:18.006873 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.006814    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q49v7\""
Jul 07 13:37:18.008158 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.008135    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-974d64977-n4fcx"]
Jul 07 13:37:18.056105 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.056073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr6br\" (UniqueName: \"kubernetes.io/projected/e39f7257-ff58-4cdd-a2d8-031c72b9322e-kube-api-access-hr6br\") pod \"authorino-974d64977-n4fcx\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") " pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.056105 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.056108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e39f7257-ff58-4cdd-a2d8-031c72b9322e-tls-cert\") pod \"authorino-974d64977-n4fcx\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") " pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.157373 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.157338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr6br\" (UniqueName: \"kubernetes.io/projected/e39f7257-ff58-4cdd-a2d8-031c72b9322e-kube-api-access-hr6br\") pod \"authorino-974d64977-n4fcx\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") " pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.157373 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.157375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e39f7257-ff58-4cdd-a2d8-031c72b9322e-tls-cert\") pod \"authorino-974d64977-n4fcx\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") " pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.159699 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.159671    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e39f7257-ff58-4cdd-a2d8-031c72b9322e-tls-cert\") pod \"authorino-974d64977-n4fcx\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") " pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.167164 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.167142    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr6br\" (UniqueName: \"kubernetes.io/projected/e39f7257-ff58-4cdd-a2d8-031c72b9322e-kube-api-access-hr6br\") pod \"authorino-974d64977-n4fcx\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") " pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.313166 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.313092    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:37:18.445318 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:18.445287    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-974d64977-n4fcx"]
Jul 07 13:37:18.448053 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:37:18.448028    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode39f7257_ff58_4cdd_a2d8_031c72b9322e.slice/crio-ff33b7144f905eb3078b855106c6c3450eb8c69e341a6639d3afb8d2295656d0 WatchSource:0}: Error finding container ff33b7144f905eb3078b855106c6c3450eb8c69e341a6639d3afb8d2295656d0: Status 404 returned error can't find the container with id ff33b7144f905eb3078b855106c6c3450eb8c69e341a6639d3afb8d2295656d0
Jul 07 13:37:19.452287 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:19.452239    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-974d64977-n4fcx" event={"ID":"e39f7257-ff58-4cdd-a2d8-031c72b9322e","Type":"ContainerStarted","Data":"ff33b7144f905eb3078b855106c6c3450eb8c69e341a6639d3afb8d2295656d0"}
Jul 07 13:37:22.467241 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:22.467203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-974d64977-n4fcx" event={"ID":"e39f7257-ff58-4cdd-a2d8-031c72b9322e","Type":"ContainerStarted","Data":"d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315"}
Jul 07 13:37:22.490433 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:22.490379    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-974d64977-n4fcx" podStartSLOduration=2.505154921 podStartE2EDuration="5.490362641s" totalImagesPullingTime="2.98520772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:37:22.488624917 +0000 UTC m=+577.021530635" watchObservedRunningTime="2026-07-07 13:37:22.490362641 +0000 UTC m=+577.023268365"
Jul 07 13:37:27.289426 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:27.289390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:27.369219 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:27.369177    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:27.438981 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:27.438945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:27.614590 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:27.614534    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:30.461315 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:30.461279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:30.644617 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:30.644579    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:38.454074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:38.454033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:37:59.603515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:37:59.603481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:00.809031 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:00.809002    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:38:01.822103 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:01.822071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:11.365052 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:11.365014    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:11.467965 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:11.467932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:21.298610 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:21.298577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:30.834940 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:30.834904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:31.250755 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:31.250720    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:38:47.305481 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:47.305428    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:38:47.933495 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:38:47.933453    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:39:04.956549 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:04.956510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:39:06.533630 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:06.533590    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:39:24.377917 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:24.377880    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:39:25.731238 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:25.731204    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:39:30.234396 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:30.234357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:39:30.307087 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:30.307052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:39:48.824957 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:48.824878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:39:50.628235 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:50.628198    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:39:52.955279 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:39:52.955244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:11.418441 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.418409    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"]
Jul 07 13:40:11.421725 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.421707    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.424509 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.424493    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 13:40:11.425554 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.425529    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 13:40:11.425658 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.425563    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 13:40:11.425712 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.425663    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gvp7x\""
Jul 07 13:40:11.438616 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.438596    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"]
Jul 07 13:40:11.464990 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.464957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:11.499214 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.499186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.499346 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.499219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.499346 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.499243    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.499420 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.499359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.499420 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.499395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rvvq\" (UniqueName: \"kubernetes.io/projected/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-kube-api-access-2rvvq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.499482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.499457    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600433 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600631 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600631 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600631 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600631 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600890 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600683    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rvvq\" (UniqueName: \"kubernetes.io/projected/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-kube-api-access-2rvvq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600952 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600890    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.600952 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.600936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.601253 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.601235    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.602800 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.602745    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.603049 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.603031    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.609016 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.608987    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rvvq\" (UniqueName: \"kubernetes.io/projected/5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48-kube-api-access-2rvvq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc\" (UID: \"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.730133 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.730045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:11.863033 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.863002    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"]
Jul 07 13:40:11.866228 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:40:11.866185    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b01bd0d_31d7_4eb2_bb8a_e7d7db75db48.slice/crio-79404c5ab398cd7870a726067d64e68ecd40cb556c47613796725576ab2d7714 WatchSource:0}: Error finding container 79404c5ab398cd7870a726067d64e68ecd40cb556c47613796725576ab2d7714: Status 404 returned error can't find the container with id 79404c5ab398cd7870a726067d64e68ecd40cb556c47613796725576ab2d7714
Jul 07 13:40:11.867428 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:11.867412    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:40:12.177138 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:12.177104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc" event={"ID":"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48","Type":"ContainerStarted","Data":"79404c5ab398cd7870a726067d64e68ecd40cb556c47613796725576ab2d7714"}
Jul 07 13:40:14.034891 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:14.034857    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:40:14.187953 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:14.187915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc" event={"ID":"5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48","Type":"ContainerStarted","Data":"4e5bb5789a985a118574bb97ac68e21407acee517c8a370c85abc89e34e7bbcb"}
Jul 07 13:40:14.209003 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:14.208956    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc" podStartSLOduration=1.9426790029999998 podStartE2EDuration="3.20893396s" totalImagesPullingTime="1.266254957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:40:14.207329293 +0000 UTC m=+748.740235021" watchObservedRunningTime="2026-07-07 13:40:14.20893396 +0000 UTC m=+748.741839708"
Jul 07 13:40:21.730472 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:21.730438    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:21.731065 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:21.730489    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:21.743009 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:21.742985    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:22.238721 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:22.238697    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc"
Jul 07 13:40:50.127135 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:50.127099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:54.091915 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.091882    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69ccf8b4bd-44dsz"]
Jul 07 13:40:54.094511 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.094493    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.102685 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.102663    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69ccf8b4bd-44dsz"]
Jul 07 13:40:54.160930 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.160902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9a2ed53-2cb5-40b3-b74f-a1063302c665-tls-cert\") pod \"authorino-69ccf8b4bd-44dsz\" (UID: \"d9a2ed53-2cb5-40b3-b74f-a1063302c665\") " pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.161054 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.160949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lr69\" (UniqueName: \"kubernetes.io/projected/d9a2ed53-2cb5-40b3-b74f-a1063302c665-kube-api-access-2lr69\") pod \"authorino-69ccf8b4bd-44dsz\" (UID: \"d9a2ed53-2cb5-40b3-b74f-a1063302c665\") " pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.261433 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.261402    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9a2ed53-2cb5-40b3-b74f-a1063302c665-tls-cert\") pod \"authorino-69ccf8b4bd-44dsz\" (UID: \"d9a2ed53-2cb5-40b3-b74f-a1063302c665\") " pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.261612 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.261460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lr69\" (UniqueName: \"kubernetes.io/projected/d9a2ed53-2cb5-40b3-b74f-a1063302c665-kube-api-access-2lr69\") pod \"authorino-69ccf8b4bd-44dsz\" (UID: \"d9a2ed53-2cb5-40b3-b74f-a1063302c665\") " pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.263697 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.263673    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9a2ed53-2cb5-40b3-b74f-a1063302c665-tls-cert\") pod \"authorino-69ccf8b4bd-44dsz\" (UID: \"d9a2ed53-2cb5-40b3-b74f-a1063302c665\") " pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.270845 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.270816    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lr69\" (UniqueName: \"kubernetes.io/projected/d9a2ed53-2cb5-40b3-b74f-a1063302c665-kube-api-access-2lr69\") pod \"authorino-69ccf8b4bd-44dsz\" (UID: \"d9a2ed53-2cb5-40b3-b74f-a1063302c665\") " pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.404054 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.404021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69ccf8b4bd-44dsz"
Jul 07 13:40:54.534066 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.534041    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69ccf8b4bd-44dsz"]
Jul 07 13:40:54.536743 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.536706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:54.537045 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:40:54.537023    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9a2ed53_2cb5_40b3_b74f_a1063302c665.slice/crio-93610e3f10ee386ee3404f801403e3143187453bba1a7a15b029c112ac76604e WatchSource:0}: Error finding container 93610e3f10ee386ee3404f801403e3143187453bba1a7a15b029c112ac76604e: Status 404 returned error can't find the container with id 93610e3f10ee386ee3404f801403e3143187453bba1a7a15b029c112ac76604e
Jul 07 13:40:54.602440 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:54.602402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:55.367327 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.367295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69ccf8b4bd-44dsz" event={"ID":"d9a2ed53-2cb5-40b3-b74f-a1063302c665","Type":"ContainerStarted","Data":"86c520ae1856f953f9c1ca38e74481ed7b3e8c45c7d6a80cac179702e5bf4ff2"}
Jul 07 13:40:55.367327 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.367330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69ccf8b4bd-44dsz" event={"ID":"d9a2ed53-2cb5-40b3-b74f-a1063302c665","Type":"ContainerStarted","Data":"93610e3f10ee386ee3404f801403e3143187453bba1a7a15b029c112ac76604e"}
Jul 07 13:40:55.385680 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.385630    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69ccf8b4bd-44dsz" podStartSLOduration=0.656654165 podStartE2EDuration="1.38561183s" totalImagesPullingTime="728.957665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:40:55.384148841 +0000 UTC m=+789.917054566" watchObservedRunningTime="2026-07-07 13:40:55.38561183 +0000 UTC m=+789.918517587"
Jul 07 13:40:55.416601 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.416571    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-974d64977-n4fcx"]
Jul 07 13:40:55.416816 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.416796    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-974d64977-n4fcx" podUID="e39f7257-ff58-4cdd-a2d8-031c72b9322e" containerName="authorino" containerID="cri-o://d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315" gracePeriod=30
Jul 07 13:40:55.665556 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.665526    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:40:55.693308 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.693261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:55.766615 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.766582    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:40:55.776823 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.776752    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr6br\" (UniqueName: \"kubernetes.io/projected/e39f7257-ff58-4cdd-a2d8-031c72b9322e-kube-api-access-hr6br\") pod \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") "
Jul 07 13:40:55.776979 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.776917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e39f7257-ff58-4cdd-a2d8-031c72b9322e-tls-cert\") pod \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\" (UID: \"e39f7257-ff58-4cdd-a2d8-031c72b9322e\") "
Jul 07 13:40:55.779572 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.779538    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39f7257-ff58-4cdd-a2d8-031c72b9322e-kube-api-access-hr6br" (OuterVolumeSpecName: "kube-api-access-hr6br") pod "e39f7257-ff58-4cdd-a2d8-031c72b9322e" (UID: "e39f7257-ff58-4cdd-a2d8-031c72b9322e"). InnerVolumeSpecName "kube-api-access-hr6br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:40:55.786932 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.786868    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39f7257-ff58-4cdd-a2d8-031c72b9322e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e39f7257-ff58-4cdd-a2d8-031c72b9322e" (UID: "e39f7257-ff58-4cdd-a2d8-031c72b9322e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:40:55.878195 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.878162    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr6br\" (UniqueName: \"kubernetes.io/projected/e39f7257-ff58-4cdd-a2d8-031c72b9322e-kube-api-access-hr6br\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:40:55.878195 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:55.878188    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e39f7257-ff58-4cdd-a2d8-031c72b9322e-tls-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:40:56.372957 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.372927    2582 generic.go:356] "Generic (PLEG): container finished" podID="e39f7257-ff58-4cdd-a2d8-031c72b9322e" containerID="d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315" exitCode=0
Jul 07 13:40:56.373440 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.372978    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-974d64977-n4fcx"
Jul 07 13:40:56.373440 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.373012    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-974d64977-n4fcx" event={"ID":"e39f7257-ff58-4cdd-a2d8-031c72b9322e","Type":"ContainerDied","Data":"d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315"}
Jul 07 13:40:56.373440 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.373055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-974d64977-n4fcx" event={"ID":"e39f7257-ff58-4cdd-a2d8-031c72b9322e","Type":"ContainerDied","Data":"ff33b7144f905eb3078b855106c6c3450eb8c69e341a6639d3afb8d2295656d0"}
Jul 07 13:40:56.373440 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.373079    2582 scope.go:117] "RemoveContainer" containerID="d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315"
Jul 07 13:40:56.383074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.383055    2582 scope.go:117] "RemoveContainer" containerID="d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315"
Jul 07 13:40:56.383367 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:40:56.383345    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315\": container with ID starting with d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315 not found: ID does not exist" containerID="d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315"
Jul 07 13:40:56.383461 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.383374    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315"} err="failed to get container status \"d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315\": rpc error: code = NotFound desc = could not find container \"d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315\": container with ID starting with d9d7c73448a1b931293b0fdffca70d8d86e79ddb673778e9d97574d2134ed315 not found: ID does not exist"
Jul 07 13:40:56.389331 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.389308    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-974d64977-n4fcx"]
Jul 07 13:40:56.391817 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:56.391796    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-974d64977-n4fcx"]
Jul 07 13:40:58.078350 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:40:58.078321    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e39f7257-ff58-4cdd-a2d8-031c72b9322e" path="/var/lib/kubelet/pods/e39f7257-ff58-4cdd-a2d8-031c72b9322e/volumes"
Jul 07 13:41:24.924794 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:41:24.924739    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:41:34.039752 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:41:34.039716    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:41:42.053405 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:41:42.053369    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:41:46.133403 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:41:46.133369    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:41:54.839043 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:41:54.839011    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:42:06.033790 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:06.033753    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:42:07.696454 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:07.696418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:09.153281 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:09.153238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:33.359623 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:33.359589    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:42:44.432242 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:44.432167    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:42:50.036563 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:50.036530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:50.131309 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:50.131265    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:50.239087 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:50.239054    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:56.757760 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:56.757719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:56.862962 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:56.862918    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:56.962491 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:56.962457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:42:57.081325 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:42:57.081237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:43:03.726733 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:03.726692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:43:05.248321 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:05.248287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:43:20.924197 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:20.924167    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:43:30.032391 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:30.032356    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:43:38.737392 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:38.737354    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:43:48.627291 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:48.627254    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:43:59.129640 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:43:59.129611    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:44:10.726265 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:44:10.726188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:44:21.878423 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:44:21.878388    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:44:28.917594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:44:28.917557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:44:31.939612 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:44:31.939580    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:44:56.738059 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:44:56.738024    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:45:09.063533 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:09.063499    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:45:34.841538 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:34.841495    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:45:44.753469 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:44.753378    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:45:51.310108 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.310071    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6977954b85-w295v"]
Jul 07 13:45:51.310605 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.310444    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39f7257-ff58-4cdd-a2d8-031c72b9322e" containerName="authorino"
Jul 07 13:45:51.310605 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.310456    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39f7257-ff58-4cdd-a2d8-031c72b9322e" containerName="authorino"
Jul 07 13:45:51.310605 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.310509    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39f7257-ff58-4cdd-a2d8-031c72b9322e" containerName="authorino"
Jul 07 13:45:51.313482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.313463    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.316141 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.316117    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 13:45:51.316285 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.316151    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 13:45:51.316285 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.316268    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 13:45:51.317311 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.317278    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4l4hj\""
Jul 07 13:45:51.317457 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.317367    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 13:45:51.317457 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.317371    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 13:45:51.317457 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.317427    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 13:45:51.317640 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.317491    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 13:45:51.321231 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.321209    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 13:45:51.325799 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.325758    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6977954b85-w295v"]
Jul 07 13:45:51.343392 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-oauth-serving-cert\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.343515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343416    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wbkz\" (UniqueName: \"kubernetes.io/projected/2ecaddb3-75b2-4683-9881-12b75da88a05-kube-api-access-6wbkz\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.343515 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343472    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-console-config\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.343627 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343543    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ecaddb3-75b2-4683-9881-12b75da88a05-console-oauth-config\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.343627 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343571    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-service-ca\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.343627 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343596    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-trusted-ca-bundle\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.343747 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.343644    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ecaddb3-75b2-4683-9881-12b75da88a05-console-serving-cert\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.385650 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.385615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:45:51.444243 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ecaddb3-75b2-4683-9881-12b75da88a05-console-oauth-config\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.444243 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-service-ca\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.444486 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-trusted-ca-bundle\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.444486 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ecaddb3-75b2-4683-9881-12b75da88a05-console-serving-cert\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.444486 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-oauth-serving-cert\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.444486 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wbkz\" (UniqueName: \"kubernetes.io/projected/2ecaddb3-75b2-4683-9881-12b75da88a05-kube-api-access-6wbkz\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.444486 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.444389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-console-config\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.445277 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.445248    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-console-config\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.445432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.445273    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-service-ca\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.445432 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.445379    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-trusted-ca-bundle\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.445524 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.445485    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ecaddb3-75b2-4683-9881-12b75da88a05-oauth-serving-cert\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.446917 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.446891    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ecaddb3-75b2-4683-9881-12b75da88a05-console-oauth-config\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.447143 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.447124    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ecaddb3-75b2-4683-9881-12b75da88a05-console-serving-cert\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.453421 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.453399    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wbkz\" (UniqueName: \"kubernetes.io/projected/2ecaddb3-75b2-4683-9881-12b75da88a05-kube-api-access-6wbkz\") pod \"console-6977954b85-w295v\" (UID: \"2ecaddb3-75b2-4683-9881-12b75da88a05\") " pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.506446 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.506410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:45:51.625559 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.625523    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:45:51.773684 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:51.773658    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6977954b85-w295v"]
Jul 07 13:45:51.776404 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:45:51.776362    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ecaddb3_75b2_4683_9881_12b75da88a05.slice/crio-19246d4da8749c50ed555b2763a9fd8077054298edb78f0b1fe35f62910dc450 WatchSource:0}: Error finding container 19246d4da8749c50ed555b2763a9fd8077054298edb78f0b1fe35f62910dc450: Status 404 returned error can't find the container with id 19246d4da8749c50ed555b2763a9fd8077054298edb78f0b1fe35f62910dc450
Jul 07 13:45:52.654631 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:52.654596    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6977954b85-w295v" event={"ID":"2ecaddb3-75b2-4683-9881-12b75da88a05","Type":"ContainerStarted","Data":"a139c462fbf60c8a92abcef721d5d4af61c8a40dd41ea2bb58ceeaad0fabcfe6"}
Jul 07 13:45:52.655079 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:52.654647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6977954b85-w295v" event={"ID":"2ecaddb3-75b2-4683-9881-12b75da88a05","Type":"ContainerStarted","Data":"19246d4da8749c50ed555b2763a9fd8077054298edb78f0b1fe35f62910dc450"}
Jul 07 13:45:52.675918 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:52.675866    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6977954b85-w295v" podStartSLOduration=1.675851691 podStartE2EDuration="1.675851691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:52.673598606 +0000 UTC m=+1087.206504351" watchObservedRunningTime="2026-07-07 13:45:52.675851691 +0000 UTC m=+1087.208757473"
Jul 07 13:45:54.234970 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:45:54.234937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:46:01.332245 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:01.332210    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:46:01.625721 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:01.625682    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:46:01.625931 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:01.625820    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:46:01.630184 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:01.630156    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:46:01.698268 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:01.698240    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6977954b85-w295v"
Jul 07 13:46:02.076874 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:02.076765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:46:02.195128 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:02.195085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:46:04.313326 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:04.313297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:46:04.464413 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:04.464378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:46:10.748544 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:10.748511    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:46:22.728903 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:22.728864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:46:33.956366 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:46:33.956329    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:47:08.463074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:47:08.463002    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:48:29.463981 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:48:29.463949    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:48:40.247510 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:48:40.247435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:48:50.555105 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:48:50.555068    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:49:01.754296 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:49:01.754258    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:49:13.769416 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:49:13.769380    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:49:23.139330 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:49:23.139296    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:49:33.040445 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:49:33.040411    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:49:43.645385 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:49:43.645352    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:49:54.434015 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:49:54.433981    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:50:03.743615 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:50:03.743584    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:50:14.199657 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:50:14.199584    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:50:38.728016 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:50:38.727978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:50:46.243749 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:50:46.243717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:50:55.742150 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:50:55.742111    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:51:06.430303 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:51:06.430266    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:51:12.782191 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:51:12.782154    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:51:14.061216 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:51:14.061171    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:51:24.030860 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:51:24.030822    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:51:38.045744 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:51:38.045666    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:51:46.881017 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:51:46.880982    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:52:01.538791 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:01.538751    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:52:11.131482 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:11.131440    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:52:21.138481 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:21.138447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:52:23.547594 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:23.547553    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:52:32.844747 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:32.844713    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:52:42.442038 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:42.441994    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:52:50.976157 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:50.976124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:52:53.543007 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:52:53.542970    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:53:05.536512 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:05.536465    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:53:13.033858 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:13.033761    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:53:25.033430 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:25.033392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:53:36.447145 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:36.447103    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:53:45.449834 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:45.449798    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:53:51.233057 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:51.233027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:51.348570 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:51.348529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:51.748438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:51.748408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:58.207731 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:58.207701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:58.310524 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:58.310492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:58.489218 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:58.489150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:58.684592 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:58.684554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:59.631344 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:59.631296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:59.741211 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:59.741183    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:59.850609 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:59.850580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:53:59.975408 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:53:59.975328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:00.339641 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:00.339553    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:00.430251 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:00.430210    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:02.299864 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:02.299829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:02.350333 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:02.350301    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:54:04.796406 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:04.796375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:04.994929 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:04.994894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:05.095813 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:05.095707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:05.372135 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:05.372053    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:08.399391 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:08.399360    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:11.364227 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:11.364190    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:54:11.975413 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:11.975379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:12.068560 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:12.068524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:12.701654 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:12.701623    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:12.813900 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:12.813867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:13.471212 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:13.471180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:13.601068 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:13.601004    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:13.705953 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:13.705921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:13.994153 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:13.994122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:14.228821 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:14.228760    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:14.374653 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:14.374623    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:15.566516 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:15.566487    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:18.501941 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:18.501906    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:19.119852 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:19.119817    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:24.689709 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:24.689679    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:24.790737 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:24.790710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:26.212678 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:26.212645    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:31.508269 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:31.508237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:31.617647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:31.617615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:31.760923 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:31.760840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:31.853568 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:31.853536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:32.999571 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:32.999540    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:33.110308 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:33.110281    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:33.213039 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:33.213001    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:33.318065 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:33.317989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:33.819630 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:33.819601    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:33.916408 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:33.916383    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:34.855263 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:34.855224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:35.255134 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:35.255101    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:54:37.302169 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:37.302097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:37.505966 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:37.505933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:37.606878 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:37.606848    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:37.814561 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:37.814523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:38.960632 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:38.960595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:44.032487 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:44.032453    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:54:44.203750 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:44.203718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:44.306732 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:44.306647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:44.701677 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:44.701650    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:44.800143 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:44.800110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:44.902468 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:44.902437    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:46.399768 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:46.399736    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:53.229060 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:53.229024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:53.343054 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:53.343011    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:53.458938 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:53.458904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:53.539173 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:53.539101    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:54:54.033957 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:54:54.033920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:20.043146 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:20.043102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:25.134882 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:25.134851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:26.605709 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:26.605671    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:26.709394 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:26.709362    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:26.846285 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:26.846254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:26.963070 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:26.962948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:43.074172 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:43.074123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:49.690625 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:49.690578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:49.780253 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:49.780204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:49.892528 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:49.892479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:55:49.981651 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:55:49.981536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:00.535619 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:00.535572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:01.399811 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:01.399740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:08.170684 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:08.170593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:08.279427 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:08.279399    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:08.392435 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:08.392392    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:08.488423 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:08.488375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:09.353810 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:09.353720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:09.457618 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:09.457577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:09.579736 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:09.579706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:11.278122 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:11.278067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:12.934438 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:12.934407    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:56:18.107584 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:18.107537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:19.468498 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:19.468446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:22.437095 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:22.437066    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:56:25.504980 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:25.504944    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:25.607333 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:25.607292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:25.872204 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:25.872171    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:26.393951 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:26.393915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:26.638983 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:26.638949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:29.627687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:29.627656    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:29.746593 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:29.746551    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:29.856498 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:29.856468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:30.407121 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:30.407092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:30.519369 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:30.519337    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:34.119933 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:34.119904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:34.827825 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:34.827760    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:34.910018 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:34.909986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:35.275606 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:35.275576    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:35.839401 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:35.839372    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:56:42.240569 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:42.240537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:42.341915 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:42.341884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:42.830257 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:42.830219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:43.208545 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:43.208517    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:43.309785 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:43.309749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:46.955687 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:46.955655    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:56:49.712865 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:49.712837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:49.789952 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:49.789916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:50.047161 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:50.047075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:50.307957 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:50.307842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:50.458054 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:50.458022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:50.598095 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:50.598066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:50.733983 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:50.733947    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:57.298590 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:57.298558    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:57.370667 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:57.370637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:57.451900 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:57.451867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:57.554408 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:57.554330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:56:57.697049 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:56:57.697015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:04.212462 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:04.212428    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:04.299956 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:04.299925    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:11.131789 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:11.131738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:11.493429 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:11.493339    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:11.594942 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:11.594899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:14.815971 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:14.815933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:15.005564 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:15.005530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:15.104003 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:15.103972    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:15.423517 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:15.423439    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:15.551175 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:15.551144    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:19.369168 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:19.369134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:20.863708 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:20.863674    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:57:32.423763 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:32.423728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:32.525927 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:32.525897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:36.313448 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:36.313415    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:36.419336 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:36.419302    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:43.937310 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.937273    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"]
Jul 07 13:57:43.940967 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.940947    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.945310 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.945285    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-52dd7e06\"/\"kube-root-ca.crt\""
Jul 07 13:57:43.945499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.945366    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-52dd7e06\"/\"e2e-lifecycle-model-52dd7e06-kserve-self-signed-certs\""
Jul 07 13:57:43.945499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.945407    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-52dd7e06\"/\"default-dockercfg-pbgqg\""
Jul 07 13:57:43.945499 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.945413    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-52dd7e06\"/\"openshift-service-ca.crt\""
Jul 07 13:57:43.951463 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.951441    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"]
Jul 07 13:57:43.964944 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.964923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-dshm\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.965060 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.964978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-tmp-dir\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.965060 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.965013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brpn4\" (UniqueName: \"kubernetes.io/projected/561053da-d59e-46bf-81e4-7417e3cfea58-kube-api-access-brpn4\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.965165 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.965081    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-model-cache\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.965165 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.965134    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561053da-d59e-46bf-81e4-7417e3cfea58-tls-certs\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.965254 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.965173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-home\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:43.980991 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:43.980951    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:44.065967 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.065935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-dshm\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066155 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.065982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-tmp-dir\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066155 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpn4\" (UniqueName: \"kubernetes.io/projected/561053da-d59e-46bf-81e4-7417e3cfea58-kube-api-access-brpn4\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066155 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-model-cache\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066155 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561053da-d59e-46bf-81e4-7417e3cfea58-tls-certs\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066383 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066176    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-home\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066383 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066368    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-tmp-dir\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066528 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066501    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-model-cache\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.066565 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.066517    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-home\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.068389 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.068371    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-dshm\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.068599 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.068581    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561053da-d59e-46bf-81e4-7417e3cfea58-tls-certs\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.074390 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.074370    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brpn4\" (UniqueName: \"kubernetes.io/projected/561053da-d59e-46bf-81e4-7417e3cfea58-kube-api-access-brpn4\") pod \"e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") " pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.250111 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.250021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:44.386494 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.386471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"]
Jul 07 13:57:44.389065 ip-10-0-129-217 kubenswrapper[2582]: W0707 13:57:44.389038    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod561053da_d59e_46bf_81e4_7417e3cfea58.slice/crio-7cb64f30b99064271832196b0752314a0dd50bf6ac6c4aecc51646c0d161063e WatchSource:0}: Error finding container 7cb64f30b99064271832196b0752314a0dd50bf6ac6c4aecc51646c0d161063e: Status 404 returned error can't find the container with id 7cb64f30b99064271832196b0752314a0dd50bf6ac6c4aecc51646c0d161063e
Jul 07 13:57:44.763623 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.763591    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw" event={"ID":"561053da-d59e-46bf-81e4-7417e3cfea58","Type":"ContainerStarted","Data":"2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7"}
Jul 07 13:57:44.763623 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.763627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw" event={"ID":"561053da-d59e-46bf-81e4-7417e3cfea58","Type":"ContainerStarted","Data":"7cb64f30b99064271832196b0752314a0dd50bf6ac6c4aecc51646c0d161063e"}
Jul 07 13:57:44.784958 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:44.784908    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw" podStartSLOduration=1.7848919319999998 podStartE2EDuration="1.784891932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:57:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:57:44.783155034 +0000 UTC m=+1799.316060770" watchObservedRunningTime="2026-07-07 13:57:44.784891932 +0000 UTC m=+1799.317797656"
Jul 07 13:57:46.650617 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:46.650580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:57:47.064543 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:47.064453    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:57:54.250211 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:54.250172    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:54.250566 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:54.250230    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:54.262483 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:54.262458    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:57:54.824854 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:57:54.824824    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:58:02.037613 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:02.037577    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:58:07.405492 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:07.405458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:58:07.904169 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:07.904136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:58:11.842903 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:11.842864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:58:13.597125 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:13.597085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:58:18.547325 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:18.547288    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"]
Jul 07 13:58:18.647753 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:18.647726    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:58:21.232952 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:21.232912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:58:33.567503 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.567445    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw" podUID="561053da-d59e-46bf-81e4-7417e3cfea58" containerName="main" containerID="cri-o://2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7" gracePeriod=45
Jul 07 13:58:33.811359 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.811336    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:58:33.905096 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brpn4\" (UniqueName: \"kubernetes.io/projected/561053da-d59e-46bf-81e4-7417e3cfea58-kube-api-access-brpn4\") pod \"561053da-d59e-46bf-81e4-7417e3cfea58\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") "
Jul 07 13:58:33.905300 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905113    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-model-cache\") pod \"561053da-d59e-46bf-81e4-7417e3cfea58\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") "
Jul 07 13:58:33.905300 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905243    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-tmp-dir\") pod \"561053da-d59e-46bf-81e4-7417e3cfea58\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") "
Jul 07 13:58:33.905300 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905290    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-home\") pod \"561053da-d59e-46bf-81e4-7417e3cfea58\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") "
Jul 07 13:58:33.905484 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905386    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-dshm\") pod \"561053da-d59e-46bf-81e4-7417e3cfea58\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") "
Jul 07 13:58:33.905484 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905381    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-model-cache" (OuterVolumeSpecName: "model-cache") pod "561053da-d59e-46bf-81e4-7417e3cfea58" (UID: "561053da-d59e-46bf-81e4-7417e3cfea58"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:58:33.905484 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905415    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561053da-d59e-46bf-81e4-7417e3cfea58-tls-certs\") pod \"561053da-d59e-46bf-81e4-7417e3cfea58\" (UID: \"561053da-d59e-46bf-81e4-7417e3cfea58\") "
Jul 07 13:58:33.905647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905466    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "561053da-d59e-46bf-81e4-7417e3cfea58" (UID: "561053da-d59e-46bf-81e4-7417e3cfea58"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:58:33.905647 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905528    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-home" (OuterVolumeSpecName: "home") pod "561053da-d59e-46bf-81e4-7417e3cfea58" (UID: "561053da-d59e-46bf-81e4-7417e3cfea58"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:58:33.905732 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905717    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:58:33.905769 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905738    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:58:33.905769 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.905749    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:58:33.907257 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.907233    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561053da-d59e-46bf-81e4-7417e3cfea58-kube-api-access-brpn4" (OuterVolumeSpecName: "kube-api-access-brpn4") pod "561053da-d59e-46bf-81e4-7417e3cfea58" (UID: "561053da-d59e-46bf-81e4-7417e3cfea58"). InnerVolumeSpecName "kube-api-access-brpn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:58:33.907745 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.907727    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-dshm" (OuterVolumeSpecName: "dshm") pod "561053da-d59e-46bf-81e4-7417e3cfea58" (UID: "561053da-d59e-46bf-81e4-7417e3cfea58"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:58:33.907846 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.907736    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/561053da-d59e-46bf-81e4-7417e3cfea58-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "561053da-d59e-46bf-81e4-7417e3cfea58" (UID: "561053da-d59e-46bf-81e4-7417e3cfea58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:58:33.986470 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.986437    2582 generic.go:356] "Generic (PLEG): container finished" podID="561053da-d59e-46bf-81e4-7417e3cfea58" containerID="2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7" exitCode=0
Jul 07 13:58:33.986634 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.986483    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"
Jul 07 13:58:33.986634 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.986527    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw" event={"ID":"561053da-d59e-46bf-81e4-7417e3cfea58","Type":"ContainerDied","Data":"2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7"}
Jul 07 13:58:33.986634 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.986574    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw" event={"ID":"561053da-d59e-46bf-81e4-7417e3cfea58","Type":"ContainerDied","Data":"7cb64f30b99064271832196b0752314a0dd50bf6ac6c4aecc51646c0d161063e"}
Jul 07 13:58:33.986634 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.986592    2582 scope.go:117] "RemoveContainer" containerID="2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7"
Jul 07 13:58:33.995258 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.995240    2582 scope.go:117] "RemoveContainer" containerID="2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7"
Jul 07 13:58:33.995498 ip-10-0-129-217 kubenswrapper[2582]: E0707 13:58:33.995480    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7\": container with ID starting with 2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7 not found: ID does not exist" containerID="2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7"
Jul 07 13:58:33.995564 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:33.995505    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7"} err="failed to get container status \"2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7\": rpc error: code = NotFound desc = could not find container \"2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7\": container with ID starting with 2670c67b012c682db4a42c53417c8f06d49744728ad1ec92d45b133d880b40a7 not found: ID does not exist"
Jul 07 13:58:34.007049 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:34.007032    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brpn4\" (UniqueName: \"kubernetes.io/projected/561053da-d59e-46bf-81e4-7417e3cfea58-kube-api-access-brpn4\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:58:34.007166 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:34.007051    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561053da-d59e-46bf-81e4-7417e3cfea58-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:58:34.007166 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:34.007061    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561053da-d59e-46bf-81e4-7417e3cfea58-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 13:58:34.009982 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:34.009962    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"]
Jul 07 13:58:34.014887 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:34.014866    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-52dd7e06/e2e-lifecycle-model-52dd7e06-kserve-7df5458dc-7n2pw"]
Jul 07 13:58:34.079074 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:34.079046    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561053da-d59e-46bf-81e4-7417e3cfea58" path="/var/lib/kubelet/pods/561053da-d59e-46bf-81e4-7417e3cfea58/volumes"
Jul 07 13:58:55.351816 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:55.351751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:58:56.878892 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:56.878814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:58:58.239534 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:58:58.239490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:59:03.035615 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:03.035578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:03.135629 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:03.135601    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:03.357705 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:03.357677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:03.617205 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:03.617116    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:04.669990 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:04.669957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:04.772460 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:04.772429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:04.928665 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:04.928575    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:05.663492 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:05.663454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:05.767244 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:05.767209    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:07.253789 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:07.253743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:08.145196 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:08.145168    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:59:13.976028 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:13.975993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:14.118547 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:14.118493    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:14.697539 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:14.697504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:21.343878 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:21.343845    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:21.444690 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:21.444658    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:21.545101 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:21.545070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:22.234530 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:22.234502    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 13:59:32.138837 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:32.138798    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:32.487517 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:32.487441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:53.474869 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:53.474834    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:59.715096 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:59.715059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 13:59:59.812982 ip-10-0-129-217 kubenswrapper[2582]: I0707 13:59:59.812947    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:00.086949 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:00.086873    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:00.420101 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:00.420064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:00.735069 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:00.734985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:03.744083 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:03.744048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:03.835654 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:03.835610    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:03.937606 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:03.937564    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:04.142328 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:04.142293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:04.399518 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:04.399425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:04.498757 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:04.498725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:08.143826 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:08.143792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:08.495677 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:08.495645    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:08.816119 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:08.816039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:10.192412 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:10.192377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:11.033865 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:11.033829    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:00:11.624692 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:11.624658    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:11.711967 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:11.711940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:11.789851 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:11.789805    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:12.108347 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:12.108318    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:12.393996 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:12.393886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:12.491134 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:12.491101    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:16.140240 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:16.140209    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:21.946599 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:21.946570    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:00:23.557983 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.557950    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"]
Jul 07 14:00:23.558369 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.558302    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561053da-d59e-46bf-81e4-7417e3cfea58" containerName="main"
Jul 07 14:00:23.558369 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.558315    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="561053da-d59e-46bf-81e4-7417e3cfea58" containerName="main"
Jul 07 14:00:23.558439 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.558384    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="561053da-d59e-46bf-81e4-7417e3cfea58" containerName="main"
Jul 07 14:00:23.561043 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.561021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.570106 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.570088    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-102141\"/\"default-dockercfg-6df5d\""
Jul 07 14:00:23.570209 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.570089    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-102141\"/\"test-model-102141-kserve-self-signed-certs\""
Jul 07 14:00:23.570209 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.570089    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-102141\"/\"openshift-service-ca.crt\""
Jul 07 14:00:23.570209 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.570092    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-102141\"/\"kube-root-ca.crt\""
Jul 07 14:00:23.578380 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.578360    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"]
Jul 07 14:00:23.581583 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.581562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:23.676415 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.676377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4q65\" (UniqueName: \"kubernetes.io/projected/293b3e88-ee34-499d-b30b-4c407cca3783-kube-api-access-s4q65\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.676601 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.676437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-dshm\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.676601 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.676453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-tmp-dir\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.676601 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.676535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/293b3e88-ee34-499d-b30b-4c407cca3783-tls-certs\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.676601 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.676557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-model-cache\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.676601 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.676583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-home\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.777649 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.777615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4q65\" (UniqueName: \"kubernetes.io/projected/293b3e88-ee34-499d-b30b-4c407cca3783-kube-api-access-s4q65\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.777851 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.777667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-dshm\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.777851 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.777687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-tmp-dir\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.777851 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.777713    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/293b3e88-ee34-499d-b30b-4c407cca3783-tls-certs\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.777851 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.777734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-model-cache\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.777851 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.777757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-home\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.778163 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.778137    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-model-cache\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.778291 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.778213    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-tmp-dir\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.778291 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.778220    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-home\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.779989 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.779973    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-dshm\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.780146 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.780131    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/293b3e88-ee34-499d-b30b-4c407cca3783-tls-certs\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.786540 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.786513    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4q65\" (UniqueName: \"kubernetes.io/projected/293b3e88-ee34-499d-b30b-4c407cca3783-kube-api-access-s4q65\") pod \"test-model-102141-kserve-6c7bc5bf5f-58tm6\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") " pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:23.869614 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:23.869589    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:24.000898 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:24.000874    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"]
Jul 07 14:00:24.003590 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:00:24.003554    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod293b3e88_ee34_499d_b30b_4c407cca3783.slice/crio-38e03ebcde1160fda79ec9cee09bfd5a3f7dec1b345cf2dd7d9be61299670ca4 WatchSource:0}: Error finding container 38e03ebcde1160fda79ec9cee09bfd5a3f7dec1b345cf2dd7d9be61299670ca4: Status 404 returned error can't find the container with id 38e03ebcde1160fda79ec9cee09bfd5a3f7dec1b345cf2dd7d9be61299670ca4
Jul 07 14:00:24.474684 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:24.474650    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6" event={"ID":"293b3e88-ee34-499d-b30b-4c407cca3783","Type":"ContainerStarted","Data":"a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07"}
Jul 07 14:00:24.474896 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:24.474692    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6" event={"ID":"293b3e88-ee34-499d-b30b-4c407cca3783","Type":"ContainerStarted","Data":"38e03ebcde1160fda79ec9cee09bfd5a3f7dec1b345cf2dd7d9be61299670ca4"}
Jul 07 14:00:24.494765 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:24.494716    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6" podStartSLOduration=1.494699491 podStartE2EDuration="1.494699491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:00:24.493504999 +0000 UTC m=+1959.026410768" watchObservedRunningTime="2026-07-07 14:00:24.494699491 +0000 UTC m=+1959.027605215"
Jul 07 14:00:25.309949 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:25.309920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:25.410985 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:25.410956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:33.207303 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:33.207268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:33.207713 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:33.207410    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:00:33.303219 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:33.303184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:33.870269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:33.870236    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:33.870269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:33.870278    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:33.882624 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:33.882596    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:34.532245 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:34.532220    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:00:52.808604 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:52.808531    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:52.908294 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:52.908266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:54.723727 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.723698    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"]
Jul 07 14:00:54.727813 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.727797    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.730570 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.730547    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-da5db0\"/\"kube-root-ca.crt\""
Jul 07 14:00:54.731694 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.731674    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-da5db0\"/\"openshift-service-ca.crt\""
Jul 07 14:00:54.731841 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.731755    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-da5db0\"/\"test-model-da5db0-kserve-self-signed-certs\""
Jul 07 14:00:54.731841 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.731817    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-da5db0\"/\"default-dockercfg-psnfq\""
Jul 07 14:00:54.738986 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.738962    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"]
Jul 07 14:00:54.743141 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.743120    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:00:54.761542 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.761509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-dshm\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.761542 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.761539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-model-cache\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.761741 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.761557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2gkl\" (UniqueName: \"kubernetes.io/projected/a5479c77-635e-4b3d-a2da-af8bd12dac8e-kube-api-access-n2gkl\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.761741 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.761682    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-home\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.761741 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.761705    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tmp-dir\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.761741 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.761729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tls-certs\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.862717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.862682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-home\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.862717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.862713    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tmp-dir\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.862959 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.862735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tls-certs\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.862959 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.862801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-dshm\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.862959 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.862824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-model-cache\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.862959 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.862849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2gkl\" (UniqueName: \"kubernetes.io/projected/a5479c77-635e-4b3d-a2da-af8bd12dac8e-kube-api-access-n2gkl\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.863139 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.863109    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-home\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.863198 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.863169    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tmp-dir\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.863275 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.863257    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-model-cache\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.865083 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.865063    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-dshm\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.865318 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.865301    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tls-certs\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:54.872636 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:54.872605    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2gkl\" (UniqueName: \"kubernetes.io/projected/a5479c77-635e-4b3d-a2da-af8bd12dac8e-kube-api-access-n2gkl\") pod \"test-model-da5db0-kserve-6646d699f7-sw5v5\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") " pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:55.037871 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:55.037735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:00:55.172942 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:55.172885    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"]
Jul 07 14:00:55.176611 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:00:55.176584    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5479c77_635e_4b3d_a2da_af8bd12dac8e.slice/crio-d82075611d9830b1ac4e7163cae5209f5e42028e659cbd7e9ef4b00e1191e3a3 WatchSource:0}: Error finding container d82075611d9830b1ac4e7163cae5209f5e42028e659cbd7e9ef4b00e1191e3a3: Status 404 returned error can't find the container with id d82075611d9830b1ac4e7163cae5209f5e42028e659cbd7e9ef4b00e1191e3a3
Jul 07 14:00:55.618530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:55.618492    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5" event={"ID":"a5479c77-635e-4b3d-a2da-af8bd12dac8e","Type":"ContainerStarted","Data":"3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e"}
Jul 07 14:00:55.618530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:55.618536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5" event={"ID":"a5479c77-635e-4b3d-a2da-af8bd12dac8e","Type":"ContainerStarted","Data":"d82075611d9830b1ac4e7163cae5209f5e42028e659cbd7e9ef4b00e1191e3a3"}
Jul 07 14:00:55.642256 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:55.642196    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5" podStartSLOduration=1.6421802570000001 podStartE2EDuration="1.642180257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:00:55.637276524 +0000 UTC m=+1990.170182249" watchObservedRunningTime="2026-07-07 14:00:55.642180257 +0000 UTC m=+1990.175085982"
Jul 07 14:00:58.643176 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:00:58.643144    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:01:03.575152 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:03.575119    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:05.038733 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:05.038693    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:01:05.039149 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:05.038749    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:01:05.052069 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:05.052039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:01:05.674979 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:05.674948    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:01:11.548496 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:11.548459    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:01:20.176744 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:20.176714    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"]
Jul 07 14:01:20.224714 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:20.224678    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:21.168417 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:21.168376    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"]
Jul 07 14:01:21.987514 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:21.987485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:22.086159 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:22.086131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:22.190706 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:22.190675    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:22.348014 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:22.347985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:22.738416 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:22.738325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:01:26.879107 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:26.879068    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:01:34.238716 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:34.238680    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:01:35.196371 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.196337    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6" podUID="293b3e88-ee34-499d-b30b-4c407cca3783" containerName="main" containerID="cri-o://a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07" gracePeriod=45
Jul 07 14:01:35.442364 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.442338    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:01:35.527275 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527184    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-home\") pod \"293b3e88-ee34-499d-b30b-4c407cca3783\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") "
Jul 07 14:01:35.527275 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527231    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/293b3e88-ee34-499d-b30b-4c407cca3783-tls-certs\") pod \"293b3e88-ee34-499d-b30b-4c407cca3783\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") "
Jul 07 14:01:35.527502 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527300    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-tmp-dir\") pod \"293b3e88-ee34-499d-b30b-4c407cca3783\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") "
Jul 07 14:01:35.527502 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527322    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-model-cache\") pod \"293b3e88-ee34-499d-b30b-4c407cca3783\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") "
Jul 07 14:01:35.527502 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527365    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4q65\" (UniqueName: \"kubernetes.io/projected/293b3e88-ee34-499d-b30b-4c407cca3783-kube-api-access-s4q65\") pod \"293b3e88-ee34-499d-b30b-4c407cca3783\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") "
Jul 07 14:01:35.527502 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527391    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-dshm\") pod \"293b3e88-ee34-499d-b30b-4c407cca3783\" (UID: \"293b3e88-ee34-499d-b30b-4c407cca3783\") "
Jul 07 14:01:35.527502 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527452    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-home" (OuterVolumeSpecName: "home") pod "293b3e88-ee34-499d-b30b-4c407cca3783" (UID: "293b3e88-ee34-499d-b30b-4c407cca3783"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:35.527731 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527567    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "293b3e88-ee34-499d-b30b-4c407cca3783" (UID: "293b3e88-ee34-499d-b30b-4c407cca3783"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:35.527731 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527607    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-model-cache" (OuterVolumeSpecName: "model-cache") pod "293b3e88-ee34-499d-b30b-4c407cca3783" (UID: "293b3e88-ee34-499d-b30b-4c407cca3783"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:35.527731 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527707    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:35.527731 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527719    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:35.527731 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.527729    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:35.529646 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.529624    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/293b3e88-ee34-499d-b30b-4c407cca3783-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "293b3e88-ee34-499d-b30b-4c407cca3783" (UID: "293b3e88-ee34-499d-b30b-4c407cca3783"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:01:35.529728 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.529655    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-dshm" (OuterVolumeSpecName: "dshm") pod "293b3e88-ee34-499d-b30b-4c407cca3783" (UID: "293b3e88-ee34-499d-b30b-4c407cca3783"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:35.529728 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.529697    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293b3e88-ee34-499d-b30b-4c407cca3783-kube-api-access-s4q65" (OuterVolumeSpecName: "kube-api-access-s4q65") pod "293b3e88-ee34-499d-b30b-4c407cca3783" (UID: "293b3e88-ee34-499d-b30b-4c407cca3783"). InnerVolumeSpecName "kube-api-access-s4q65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:01:35.629018 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.628982    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/293b3e88-ee34-499d-b30b-4c407cca3783-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:35.629018 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.629012    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4q65\" (UniqueName: \"kubernetes.io/projected/293b3e88-ee34-499d-b30b-4c407cca3783-kube-api-access-s4q65\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:35.629018 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.629025    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/293b3e88-ee34-499d-b30b-4c407cca3783-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:35.803897 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.803802    2582 generic.go:356] "Generic (PLEG): container finished" podID="293b3e88-ee34-499d-b30b-4c407cca3783" containerID="a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07" exitCode=0
Jul 07 14:01:35.803897 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.803837    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"
Jul 07 14:01:35.803897 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.803883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6" event={"ID":"293b3e88-ee34-499d-b30b-4c407cca3783","Type":"ContainerDied","Data":"a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07"}
Jul 07 14:01:35.804122 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.803926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6" event={"ID":"293b3e88-ee34-499d-b30b-4c407cca3783","Type":"ContainerDied","Data":"38e03ebcde1160fda79ec9cee09bfd5a3f7dec1b345cf2dd7d9be61299670ca4"}
Jul 07 14:01:35.804122 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.803949    2582 scope.go:117] "RemoveContainer" containerID="a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07"
Jul 07 14:01:35.813100 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.813076    2582 scope.go:117] "RemoveContainer" containerID="a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07"
Jul 07 14:01:35.813344 ip-10-0-129-217 kubenswrapper[2582]: E0707 14:01:35.813325    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07\": container with ID starting with a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07 not found: ID does not exist" containerID="a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07"
Jul 07 14:01:35.813414 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.813358    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07"} err="failed to get container status \"a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07\": rpc error: code = NotFound desc = could not find container \"a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07\": container with ID starting with a99e008377026634502e0136e9c81ef1949593e026a84bdf91f0ab5c15ba3d07 not found: ID does not exist"
Jul 07 14:01:35.825298 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.825275    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"]
Jul 07 14:01:35.834082 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:35.832320    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-102141/test-model-102141-kserve-6c7bc5bf5f-58tm6"]
Jul 07 14:01:36.079372 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.079281    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="293b3e88-ee34-499d-b30b-4c407cca3783" path="/var/lib/kubelet/pods/293b3e88-ee34-499d-b30b-4c407cca3783/volumes"
Jul 07 14:01:36.188203 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.188167    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5" podUID="a5479c77-635e-4b3d-a2da-af8bd12dac8e" containerName="main" containerID="cri-o://3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e" gracePeriod=45
Jul 07 14:01:36.435613 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.435590    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:01:36.540523 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540492    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-model-cache\") pod \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") "
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540541    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2gkl\" (UniqueName: \"kubernetes.io/projected/a5479c77-635e-4b3d-a2da-af8bd12dac8e-kube-api-access-n2gkl\") pod \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") "
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540571    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-dshm\") pod \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") "
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540588    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tmp-dir\") pod \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") "
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540668    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tls-certs\") pod \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") "
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540692    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-home\") pod \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\" (UID: \"a5479c77-635e-4b3d-a2da-af8bd12dac8e\") "
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540750    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5479c77-635e-4b3d-a2da-af8bd12dac8e" (UID: "a5479c77-635e-4b3d-a2da-af8bd12dac8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.540983    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5479c77-635e-4b3d-a2da-af8bd12dac8e" (UID: "a5479c77-635e-4b3d-a2da-af8bd12dac8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.541001    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-home" (OuterVolumeSpecName: "home") pod "a5479c77-635e-4b3d-a2da-af8bd12dac8e" (UID: "a5479c77-635e-4b3d-a2da-af8bd12dac8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:36.541137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.541039    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:36.542898 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.542874    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5479c77-635e-4b3d-a2da-af8bd12dac8e-kube-api-access-n2gkl" (OuterVolumeSpecName: "kube-api-access-n2gkl") pod "a5479c77-635e-4b3d-a2da-af8bd12dac8e" (UID: "a5479c77-635e-4b3d-a2da-af8bd12dac8e"). InnerVolumeSpecName "kube-api-access-n2gkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:01:36.542994 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.542898    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5479c77-635e-4b3d-a2da-af8bd12dac8e" (UID: "a5479c77-635e-4b3d-a2da-af8bd12dac8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:01:36.542994 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.542906    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-dshm" (OuterVolumeSpecName: "dshm") pod "a5479c77-635e-4b3d-a2da-af8bd12dac8e" (UID: "a5479c77-635e-4b3d-a2da-af8bd12dac8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:01:36.641530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.641444    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:36.641530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.641494    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:36.641530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.641507    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5479c77-635e-4b3d-a2da-af8bd12dac8e-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:36.641530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.641514    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5479c77-635e-4b3d-a2da-af8bd12dac8e-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:36.641530 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.641524    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2gkl\" (UniqueName: \"kubernetes.io/projected/a5479c77-635e-4b3d-a2da-af8bd12dac8e-kube-api-access-n2gkl\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:01:36.809880 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.809843    2582 generic.go:356] "Generic (PLEG): container finished" podID="a5479c77-635e-4b3d-a2da-af8bd12dac8e" containerID="3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e" exitCode=0
Jul 07 14:01:36.810067 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.809890    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"
Jul 07 14:01:36.810067 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.809930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5" event={"ID":"a5479c77-635e-4b3d-a2da-af8bd12dac8e","Type":"ContainerDied","Data":"3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e"}
Jul 07 14:01:36.810067 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.809972    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5" event={"ID":"a5479c77-635e-4b3d-a2da-af8bd12dac8e","Type":"ContainerDied","Data":"d82075611d9830b1ac4e7163cae5209f5e42028e659cbd7e9ef4b00e1191e3a3"}
Jul 07 14:01:36.810067 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.809990    2582 scope.go:117] "RemoveContainer" containerID="3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e"
Jul 07 14:01:36.818656 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.818639    2582 scope.go:117] "RemoveContainer" containerID="3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e"
Jul 07 14:01:36.818903 ip-10-0-129-217 kubenswrapper[2582]: E0707 14:01:36.818885    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e\": container with ID starting with 3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e not found: ID does not exist" containerID="3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e"
Jul 07 14:01:36.818957 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.818910    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e"} err="failed to get container status \"3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e\": rpc error: code = NotFound desc = could not find container \"3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e\": container with ID starting with 3981a0932eef84f4fe0c3b6e50224927aa8f77285bbb268ea1bdc8992cf0fb2e not found: ID does not exist"
Jul 07 14:01:36.833422 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.833395    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"]
Jul 07 14:01:36.835103 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:36.835082    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-da5db0/test-model-da5db0-kserve-6646d699f7-sw5v5"]
Jul 07 14:01:38.078263 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:01:38.078229    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5479c77-635e-4b3d-a2da-af8bd12dac8e" path="/var/lib/kubelet/pods/a5479c77-635e-4b3d-a2da-af8bd12dac8e/volumes"
Jul 07 14:02:04.647487 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:04.647456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:04.945179 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:04.945071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:05.172269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:05.172233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:05.327875 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:05.327765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:05.815273 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:05.815243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:05.939397 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:05.939366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:06.049470 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:06.049441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:06.192058 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:06.192030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:06.340262 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:06.340234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:09.141318 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:09.141282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:02:12.713268 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:12.713238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:12.857972 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:12.857940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:13.003578 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:13.003503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:13.238631 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:13.238600    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:13.614009 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:13.613977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:13.988752 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:13.988665    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:14.284158 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:14.284072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:17.412594 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:17.412557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:17.568052 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:17.568022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:17.695752 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:17.695663    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:18.164077 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:18.164042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:18.255160 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:18.255115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:21.918603 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:21.918556    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:22.437945 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:22.437909    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:22.825825 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:22.825719    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:02:22.864482 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:22.864446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:23.092811 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:23.092715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:25.680723 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:25.680683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:25.819010 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:25.818969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:25.933558 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:25.933470    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:26.158600 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:26.158572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:26.579252 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:26.579220    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:26.679622 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:26.679584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:30.164626 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:30.164592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:32.674450 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:32.674415    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:02:33.754171 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:33.754137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:35.442572 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:35.442538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:38.633750 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:38.633712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:39.122351 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:39.122319    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:39.450370 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:39.450298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:39.555671 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:39.555640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:39.938820 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:39.938788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:45.475824 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:45.475789    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:02:47.820506 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:47.820475    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:47.915252 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:47.915216    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:48.604900 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:48.604872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:48.700916 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:48.700889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:49.007943 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:49.007854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:49.329054 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:49.328977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:49.500921 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:49.500893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:49.955501 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:49.955468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:50.223091 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:50.223012    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:57.380453 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:57.380421    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:57.501611 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:57.501583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:57.734539 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:57.734450    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:58.005598 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:58.005511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:58.353974 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:58.353942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:58.373700 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:58.373673    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:02:58.737340 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:58.737250    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:02:59.311581 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:02:59.311547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:01.804737 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:01.804705    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:02.026954 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:02.026920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:02.232644 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:02.232614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:02.865274 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:02.865237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:03.166117 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:03.166030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:03.423629 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:03.423542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:06.336754 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:06.336720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:06.778457 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:06.778424    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:07.350063 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:07.350032    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:07.730720 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:07.730635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:09.485361 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:09.485326    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:03:10.841069 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:10.841033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:11.056589 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:11.056555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:11.196470 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:11.196441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:11.386581 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:11.386552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:14.221592 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:14.221562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:14.312557 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:14.312528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:15.322966 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:15.322936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:22.269971 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:22.269937    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:03:24.430159 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:24.430126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:24.577209 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:24.577174    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:33.793734 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:33.793699    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:33.930058 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:33.930028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:39.235729 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.235657    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"]
Jul 07 14:03:39.236138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.236069    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293b3e88-ee34-499d-b30b-4c407cca3783" containerName="main"
Jul 07 14:03:39.236138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.236081    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="293b3e88-ee34-499d-b30b-4c407cca3783" containerName="main"
Jul 07 14:03:39.236138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.236093    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5479c77-635e-4b3d-a2da-af8bd12dac8e" containerName="main"
Jul 07 14:03:39.236138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.236102    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5479c77-635e-4b3d-a2da-af8bd12dac8e" containerName="main"
Jul 07 14:03:39.236269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.236170    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5479c77-635e-4b3d-a2da-af8bd12dac8e" containerName="main"
Jul 07 14:03:39.236269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.236182    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="293b3e88-ee34-499d-b30b-4c407cca3783" containerName="main"
Jul 07 14:03:39.239033 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.239018    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.245604 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.245141    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b6630e\"/\"openshift-service-ca.crt\""
Jul 07 14:03:39.245604 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.245171    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b6630e\"/\"auth-test-model-b6630e-kserve-self-signed-certs\""
Jul 07 14:03:39.245604 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.245183    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b6630e\"/\"kube-root-ca.crt\""
Jul 07 14:03:39.245604 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.245171    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b6630e\"/\"default-dockercfg-dsl2d\""
Jul 07 14:03:39.255991 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.255964    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"]
Jul 07 14:03:39.257978 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.257950    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:03:39.278692 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.278667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-model-cache\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.278692 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.278694    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-tmp-dir\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.278885 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.278752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf79m\" (UniqueName: \"kubernetes.io/projected/2d1cd14f-1a24-479e-9515-415033bd49f3-kube-api-access-lf79m\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.278885 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.278837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-dshm\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.278959 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.278908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-home\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.278959 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.278932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1cd14f-1a24-479e-9515-415033bd49f3-tls-certs\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.379510 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-dshm\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.379717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-home\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.379717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379580    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1cd14f-1a24-479e-9515-415033bd49f3-tls-certs\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.379717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379607    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-model-cache\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.379717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-tmp-dir\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.379717 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf79m\" (UniqueName: \"kubernetes.io/projected/2d1cd14f-1a24-479e-9515-415033bd49f3-kube-api-access-lf79m\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.380016 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.379988    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-home\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.380089 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.380072    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-tmp-dir\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.380132 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.380070    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-model-cache\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.381881 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.381861    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-dshm\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.381967 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.381958    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1cd14f-1a24-479e-9515-415033bd49f3-tls-certs\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.389958 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.389937    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf79m\" (UniqueName: \"kubernetes.io/projected/2d1cd14f-1a24-479e-9515-415033bd49f3-kube-api-access-lf79m\") pod \"auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") " pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.548369 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.548275    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:39.688120 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:39.688090    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"]
Jul 07 14:03:39.688948 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:03:39.688921    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d1cd14f_1a24_479e_9515_415033bd49f3.slice/crio-bda0d83108b5e70cebbe01ef9c81a92c23073c6362feb1cf78ecdab5f350ff0c WatchSource:0}: Error finding container bda0d83108b5e70cebbe01ef9c81a92c23073c6362feb1cf78ecdab5f350ff0c: Status 404 returned error can't find the container with id bda0d83108b5e70cebbe01ef9c81a92c23073c6362feb1cf78ecdab5f350ff0c
Jul 07 14:03:40.364836 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:40.364796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw" event={"ID":"2d1cd14f-1a24-479e-9515-415033bd49f3","Type":"ContainerStarted","Data":"d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084"}
Jul 07 14:03:40.364836 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:40.364839    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw" event={"ID":"2d1cd14f-1a24-479e-9515-415033bd49f3","Type":"ContainerStarted","Data":"bda0d83108b5e70cebbe01ef9c81a92c23073c6362feb1cf78ecdab5f350ff0c"}
Jul 07 14:03:40.390149 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:40.390105    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw" podStartSLOduration=1.390090663 podStartE2EDuration="1.390090663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:03:40.388640097 +0000 UTC m=+2154.921545823" watchObservedRunningTime="2026-07-07 14:03:40.390090663 +0000 UTC m=+2154.922996388"
Jul 07 14:03:43.474714 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:43.474682    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:03:49.548635 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:49.548600    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:49.551011 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:49.548665    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:49.561194 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:49.561175    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:50.420280 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:50.420250    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:03:56.668720 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:03:56.668684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:04:00.640429 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.640393    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"]
Jul 07 14:04:00.643672 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.643657    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.648370 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.648347    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"kube-root-ca.crt\""
Jul 07 14:04:00.649590 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.649569    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"default-dockercfg-lbv8g\""
Jul 07 14:04:00.649693 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.649591    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"openshift-service-ca.crt\""
Jul 07 14:04:00.649693 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.649608    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"auth-test-model-f0b1df-kserve-self-signed-certs\""
Jul 07 14:04:00.677368 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.677339    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"]
Jul 07 14:04:00.698499 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.698458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:04:00.762600 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.762567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tmp-dir\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.762810 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.762614    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-model-cache\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.762810 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.762667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tls-certs\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.762810 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.762717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-home\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.762810 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.762750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-dshm\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.762810 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.762768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv5f7\" (UniqueName: \"kubernetes.io/projected/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-kube-api-access-nv5f7\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.863583 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.863545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-home\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.863757 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.863600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-dshm\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.863757 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.863709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv5f7\" (UniqueName: \"kubernetes.io/projected/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-kube-api-access-nv5f7\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.863863 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.863755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tmp-dir\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.863863 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.863846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-model-cache\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.863934 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.863893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tls-certs\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.864047 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.864029    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-home\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.864169 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.864150    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tmp-dir\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.864285 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.864192    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-model-cache\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.865927 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.865896    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-dshm\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.866225 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.866198    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tls-certs\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.880256 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.880230    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv5f7\" (UniqueName: \"kubernetes.io/projected/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-kube-api-access-nv5f7\") pod \"auth-test-model-f0b1df-kserve-75bddb9c55-w62v5\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") " pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:00.952997 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:00.952907    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:01.107385 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:01.107352    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"]
Jul 07 14:04:01.110515 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:04:01.110487    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1aaa7c94_c5bd_465c_ab0d_debad3dab19f.slice/crio-35515cc4fe08589a95922cd3a06276ee2bee47211f13a679c866d0336c4c326f WatchSource:0}: Error finding container 35515cc4fe08589a95922cd3a06276ee2bee47211f13a679c866d0336c4c326f: Status 404 returned error can't find the container with id 35515cc4fe08589a95922cd3a06276ee2bee47211f13a679c866d0336c4c326f
Jul 07 14:04:01.460220 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:01.460182    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5" event={"ID":"1aaa7c94-c5bd-465c-ab0d-debad3dab19f","Type":"ContainerStarted","Data":"ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117"}
Jul 07 14:04:01.460424 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:01.460228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5" event={"ID":"1aaa7c94-c5bd-465c-ab0d-debad3dab19f","Type":"ContainerStarted","Data":"35515cc4fe08589a95922cd3a06276ee2bee47211f13a679c866d0336c4c326f"}
Jul 07 14:04:04.158577 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:04.158523    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5" podStartSLOduration=4.158505007 podStartE2EDuration="4.158505007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:04:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:04:01.48984866 +0000 UTC m=+2176.022754384" watchObservedRunningTime="2026-07-07 14:04:04.158505007 +0000 UTC m=+2178.691410731"
Jul 07 14:04:04.159478 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:04.159459    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:04:10.953057 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:10.953024    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:10.953518 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:10.953074    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:10.965332 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:10.965311    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:11.515055 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:11.515026    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:04:17.359725 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:17.359689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:04:18.241375 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:18.241340    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:04:34.452371 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:34.452327    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:04:48.759524 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:04:48.759491    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:05:05.840403 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:05.840363    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:05:12.644592 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:12.644563    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:05:12.745265 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:12.745234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:05:21.337716 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:21.337685    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:05:38.934729 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:38.934696    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:05:50.604452 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:50.604416    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"]
Jul 07 14:05:50.614453 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:50.614429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:05:51.528097 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:51.528064    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"]
Jul 07 14:05:51.551655 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:51.551626    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:05:54.630844 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:05:54.630812    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:06:03.461000 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.460967    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"]
Jul 07 14:06:03.464630 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.464614    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.467836 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.467818    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b6630e\"/\"sub-test-model-b6630e-kserve-self-signed-certs\""
Jul 07 14:06:03.484957 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.484928    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"]
Jul 07 14:06:03.493336 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.493310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:06:03.586623 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.586591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-dshm\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.586623 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.586627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tmp-dir\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.586902 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.586659    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbkmp\" (UniqueName: \"kubernetes.io/projected/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-kube-api-access-xbkmp\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.586902 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.586690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tls-certs\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.586902 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.586717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-home\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.586902 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.586735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-model-cache\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.687930 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.687890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbkmp\" (UniqueName: \"kubernetes.io/projected/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-kube-api-access-xbkmp\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688129 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.687948    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tls-certs\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688129 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.687979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-home\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688129 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.687997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-model-cache\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688129 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.688085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-dshm\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688129 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.688112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tmp-dir\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688470 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.688448    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-home\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688539 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.688487    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-model-cache\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.688539 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.688522    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tmp-dir\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.690325 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.690301    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-dshm\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.690720 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.690698    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tls-certs\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.698609 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.698581    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbkmp\" (UniqueName: \"kubernetes.io/projected/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-kube-api-access-xbkmp\") pod \"sub-test-model-b6630e-kserve-69d7957b7-zn8gc\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") " pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.773760 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.773658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:03.910641 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.910613    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"]
Jul 07 14:06:03.913041 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:06:03.913008    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d9331c1_468b_46a9_898d_fbe2b4cb41ac.slice/crio-73899992b8c71ce3f6977c0b5ee347faf4e028f764852bc0202ddad3f9b41091 WatchSource:0}: Error finding container 73899992b8c71ce3f6977c0b5ee347faf4e028f764852bc0202ddad3f9b41091: Status 404 returned error can't find the container with id 73899992b8c71ce3f6977c0b5ee347faf4e028f764852bc0202ddad3f9b41091
Jul 07 14:06:03.999664 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.999627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc" event={"ID":"3d9331c1-468b-46a9-898d-fbe2b4cb41ac","Type":"ContainerStarted","Data":"209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b"}
Jul 07 14:06:03.999885 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:03.999672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc" event={"ID":"3d9331c1-468b-46a9-898d-fbe2b4cb41ac","Type":"ContainerStarted","Data":"73899992b8c71ce3f6977c0b5ee347faf4e028f764852bc0202ddad3f9b41091"}
Jul 07 14:06:04.025353 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:04.025247    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc" podStartSLOduration=1.025228794 podStartE2EDuration="1.025228794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:06:04.024161108 +0000 UTC m=+2298.557066862" watchObservedRunningTime="2026-07-07 14:06:04.025228794 +0000 UTC m=+2298.558134519"
Jul 07 14:06:05.624262 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:05.624224    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw" podUID="2d1cd14f-1a24-479e-9515-415033bd49f3" containerName="main" containerID="cri-o://d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084" gracePeriod=45
Jul 07 14:06:05.870288 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:05.870267    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:06:06.009684 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.009586    2582 generic.go:356] "Generic (PLEG): container finished" podID="2d1cd14f-1a24-479e-9515-415033bd49f3" containerID="d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084" exitCode=0
Jul 07 14:06:06.009684 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.009630    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"
Jul 07 14:06:06.009905 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.009682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw" event={"ID":"2d1cd14f-1a24-479e-9515-415033bd49f3","Type":"ContainerDied","Data":"d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084"}
Jul 07 14:06:06.009905 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.009721    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw" event={"ID":"2d1cd14f-1a24-479e-9515-415033bd49f3","Type":"ContainerDied","Data":"bda0d83108b5e70cebbe01ef9c81a92c23073c6362feb1cf78ecdab5f350ff0c"}
Jul 07 14:06:06.009905 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.009741    2582 scope.go:117] "RemoveContainer" containerID="d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084"
Jul 07 14:06:06.010251 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010233    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf79m\" (UniqueName: \"kubernetes.io/projected/2d1cd14f-1a24-479e-9515-415033bd49f3-kube-api-access-lf79m\") pod \"2d1cd14f-1a24-479e-9515-415033bd49f3\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") "
Jul 07 14:06:06.010313 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010275    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-tmp-dir\") pod \"2d1cd14f-1a24-479e-9515-415033bd49f3\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") "
Jul 07 14:06:06.010362 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010321    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-model-cache\") pod \"2d1cd14f-1a24-479e-9515-415033bd49f3\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") "
Jul 07 14:06:06.010362 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010341    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1cd14f-1a24-479e-9515-415033bd49f3-tls-certs\") pod \"2d1cd14f-1a24-479e-9515-415033bd49f3\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") "
Jul 07 14:06:06.010588 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010570    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-dshm\") pod \"2d1cd14f-1a24-479e-9515-415033bd49f3\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") "
Jul 07 14:06:06.010689 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-home\") pod \"2d1cd14f-1a24-479e-9515-415033bd49f3\" (UID: \"2d1cd14f-1a24-479e-9515-415033bd49f3\") "
Jul 07 14:06:06.010689 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010580    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d1cd14f-1a24-479e-9515-415033bd49f3" (UID: "2d1cd14f-1a24-479e-9515-415033bd49f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.010689 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010595    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d1cd14f-1a24-479e-9515-415033bd49f3" (UID: "2d1cd14f-1a24-479e-9515-415033bd49f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.010932 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.010911    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-home" (OuterVolumeSpecName: "home") pod "2d1cd14f-1a24-479e-9515-415033bd49f3" (UID: "2d1cd14f-1a24-479e-9515-415033bd49f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.011068 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.011049    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.011144 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.011074    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.011144 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.011087    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.012660 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.012640    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1cd14f-1a24-479e-9515-415033bd49f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d1cd14f-1a24-479e-9515-415033bd49f3" (UID: "2d1cd14f-1a24-479e-9515-415033bd49f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:06:06.012740 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.012707    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-dshm" (OuterVolumeSpecName: "dshm") pod "2d1cd14f-1a24-479e-9515-415033bd49f3" (UID: "2d1cd14f-1a24-479e-9515-415033bd49f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.012828 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.012800    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1cd14f-1a24-479e-9515-415033bd49f3-kube-api-access-lf79m" (OuterVolumeSpecName: "kube-api-access-lf79m") pod "2d1cd14f-1a24-479e-9515-415033bd49f3" (UID: "2d1cd14f-1a24-479e-9515-415033bd49f3"). InnerVolumeSpecName "kube-api-access-lf79m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:06:06.029531 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.029512    2582 scope.go:117] "RemoveContainer" containerID="d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084"
Jul 07 14:06:06.029880 ip-10-0-129-217 kubenswrapper[2582]: E0707 14:06:06.029846    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084\": container with ID starting with d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084 not found: ID does not exist" containerID="d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084"
Jul 07 14:06:06.029973 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.029877    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084"} err="failed to get container status \"d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084\": rpc error: code = NotFound desc = could not find container \"d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084\": container with ID starting with d8e36cadbf13d66ae6d1bd42aa1a1d94cb2f6533404f7daef9501290768c4084 not found: ID does not exist"
Jul 07 14:06:06.112138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.112111    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf79m\" (UniqueName: \"kubernetes.io/projected/2d1cd14f-1a24-479e-9515-415033bd49f3-kube-api-access-lf79m\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.112138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.112134    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1cd14f-1a24-479e-9515-415033bd49f3-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.112138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.112143    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1cd14f-1a24-479e-9515-415033bd49f3-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.331110 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.331017    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"]
Jul 07 14:06:06.335448 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.335421    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/auth-test-model-b6630e-kserve-d87ffd4f9-cwsdw"]
Jul 07 14:06:06.547701 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.547660    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5" podUID="1aaa7c94-c5bd-465c-ab0d-debad3dab19f" containerName="main" containerID="cri-o://ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117" gracePeriod=45
Jul 07 14:06:06.789914 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.789890    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:06:06.919895 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.919821    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-home\") pod \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") "
Jul 07 14:06:06.919895 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.919869    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-model-cache\") pod \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") "
Jul 07 14:06:06.919895 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.919887    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv5f7\" (UniqueName: \"kubernetes.io/projected/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-kube-api-access-nv5f7\") pod \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") "
Jul 07 14:06:06.920111 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.919928    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tls-certs\") pod \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") "
Jul 07 14:06:06.920111 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tmp-dir\") pod \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") "
Jul 07 14:06:06.920111 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920048    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-dshm\") pod \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\" (UID: \"1aaa7c94-c5bd-465c-ab0d-debad3dab19f\") "
Jul 07 14:06:06.920269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920104    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-home" (OuterVolumeSpecName: "home") pod "1aaa7c94-c5bd-465c-ab0d-debad3dab19f" (UID: "1aaa7c94-c5bd-465c-ab0d-debad3dab19f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.920269 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920120    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-model-cache" (OuterVolumeSpecName: "model-cache") pod "1aaa7c94-c5bd-465c-ab0d-debad3dab19f" (UID: "1aaa7c94-c5bd-465c-ab0d-debad3dab19f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.920378 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920294    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1aaa7c94-c5bd-465c-ab0d-debad3dab19f" (UID: "1aaa7c94-c5bd-465c-ab0d-debad3dab19f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.920430 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920414    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.920486 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920456    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.920486 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.920470    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:06.922166 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.922147    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-dshm" (OuterVolumeSpecName: "dshm") pod "1aaa7c94-c5bd-465c-ab0d-debad3dab19f" (UID: "1aaa7c94-c5bd-465c-ab0d-debad3dab19f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:06:06.922257 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.922229    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1aaa7c94-c5bd-465c-ab0d-debad3dab19f" (UID: "1aaa7c94-c5bd-465c-ab0d-debad3dab19f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:06:06.922257 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:06.922239    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-kube-api-access-nv5f7" (OuterVolumeSpecName: "kube-api-access-nv5f7") pod "1aaa7c94-c5bd-465c-ab0d-debad3dab19f" (UID: "1aaa7c94-c5bd-465c-ab0d-debad3dab19f"). InnerVolumeSpecName "kube-api-access-nv5f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:06:07.015749 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.015718    2582 generic.go:356] "Generic (PLEG): container finished" podID="1aaa7c94-c5bd-465c-ab0d-debad3dab19f" containerID="ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117" exitCode=0
Jul 07 14:06:07.015960 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.015763    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"
Jul 07 14:06:07.015960 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.015809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5" event={"ID":"1aaa7c94-c5bd-465c-ab0d-debad3dab19f","Type":"ContainerDied","Data":"ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117"}
Jul 07 14:06:07.015960 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.015858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5" event={"ID":"1aaa7c94-c5bd-465c-ab0d-debad3dab19f","Type":"ContainerDied","Data":"35515cc4fe08589a95922cd3a06276ee2bee47211f13a679c866d0336c4c326f"}
Jul 07 14:06:07.015960 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.015880    2582 scope.go:117] "RemoveContainer" containerID="ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117"
Jul 07 14:06:07.021051 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.021027    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv5f7\" (UniqueName: \"kubernetes.io/projected/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-kube-api-access-nv5f7\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:07.021051 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.021049    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:07.021240 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.021059    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aaa7c94-c5bd-465c-ab0d-debad3dab19f-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:06:07.025033 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.025017    2582 scope.go:117] "RemoveContainer" containerID="ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117"
Jul 07 14:06:07.025289 ip-10-0-129-217 kubenswrapper[2582]: E0707 14:06:07.025270    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117\": container with ID starting with ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117 not found: ID does not exist" containerID="ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117"
Jul 07 14:06:07.025336 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.025298    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117"} err="failed to get container status \"ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117\": rpc error: code = NotFound desc = could not find container \"ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117\": container with ID starting with ae5ad8f1772717ae82eb583a9e18f380cedd8a5aaba46e715adbe646adc7b117 not found: ID does not exist"
Jul 07 14:06:07.039633 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.039608    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"]
Jul 07 14:06:07.043948 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.043926    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/auth-test-model-f0b1df-kserve-75bddb9c55-w62v5"]
Jul 07 14:06:07.949238 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:07.949206    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:06:08.077679 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:08.077646    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aaa7c94-c5bd-465c-ab0d-debad3dab19f" path="/var/lib/kubelet/pods/1aaa7c94-c5bd-465c-ab0d-debad3dab19f/volumes"
Jul 07 14:06:08.078070 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:08.078056    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1cd14f-1a24-479e-9515-415033bd49f3" path="/var/lib/kubelet/pods/2d1cd14f-1a24-479e-9515-415033bd49f3/volumes"
Jul 07 14:06:13.774205 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:13.774167    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:13.774668 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:13.774230    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:13.787166 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:13.787141    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:14.060380 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:14.060300    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:06:25.927460 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927431    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"]
Jul 07 14:06:25.927947 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927811    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aaa7c94-c5bd-465c-ab0d-debad3dab19f" containerName="main"
Jul 07 14:06:25.927947 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927827    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aaa7c94-c5bd-465c-ab0d-debad3dab19f" containerName="main"
Jul 07 14:06:25.927947 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927844    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1cd14f-1a24-479e-9515-415033bd49f3" containerName="main"
Jul 07 14:06:25.927947 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927849    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1cd14f-1a24-479e-9515-415033bd49f3" containerName="main"
Jul 07 14:06:25.927947 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927907    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1cd14f-1a24-479e-9515-415033bd49f3" containerName="main"
Jul 07 14:06:25.927947 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.927917    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aaa7c94-c5bd-465c-ab0d-debad3dab19f" containerName="main"
Jul 07 14:06:25.930874 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.930850    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:25.934144 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.934123    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"kube-root-ca.crt\""
Jul 07 14:06:25.938789 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.935246    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"openshift-service-ca.crt\""
Jul 07 14:06:25.938789 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.935245    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"default-dockercfg-lbv8g\""
Jul 07 14:06:25.938789 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.935565    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"sub-test-model-f0b1df-kserve-self-signed-certs\""
Jul 07 14:06:25.944026 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.944001    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"]
Jul 07 14:06:25.960052 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.960027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:06:25.992254 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.992232    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-model-cache\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:25.992377 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.992302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-home\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:25.992377 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.992325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2l5k\" (UniqueName: \"kubernetes.io/projected/1e402935-ca7d-4a1d-8d70-d244de61ca19-kube-api-access-r2l5k\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:25.992377 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.992347    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e402935-ca7d-4a1d-8d70-d244de61ca19-tls-certs\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:25.992520 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.992402    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-tmp-dir\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:25.992520 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:25.992453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-dshm\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093139 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-dshm\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093139 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-model-cache\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093371 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-home\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093371 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2l5k\" (UniqueName: \"kubernetes.io/projected/1e402935-ca7d-4a1d-8d70-d244de61ca19-kube-api-access-r2l5k\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093371 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e402935-ca7d-4a1d-8d70-d244de61ca19-tls-certs\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093371 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-tmp-dir\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093599 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093569    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-model-cache\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093866 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093841    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-tmp-dir\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.093866 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.093850    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-home\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.095392 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.095374    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-dshm\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.095625 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.095606    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e402935-ca7d-4a1d-8d70-d244de61ca19-tls-certs\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.102997 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.102977    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2l5k\" (UniqueName: \"kubernetes.io/projected/1e402935-ca7d-4a1d-8d70-d244de61ca19-kube-api-access-r2l5k\") pod \"sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") " pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.242804 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.242691    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:26.384279 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:26.384246    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"]
Jul 07 14:06:26.388059 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:06:26.388032    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e402935_ca7d_4a1d_8d70_d244de61ca19.slice/crio-60eb5b4026b7cdfd8cacc0cd655e16306f89efa5f2fd28a511d879c1040dc81e WatchSource:0}: Error finding container 60eb5b4026b7cdfd8cacc0cd655e16306f89efa5f2fd28a511d879c1040dc81e: Status 404 returned error can't find the container with id 60eb5b4026b7cdfd8cacc0cd655e16306f89efa5f2fd28a511d879c1040dc81e
Jul 07 14:06:27.107571 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:27.107534    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp" event={"ID":"1e402935-ca7d-4a1d-8d70-d244de61ca19","Type":"ContainerStarted","Data":"531fdabce81f546b3ffb8dd9b21a68bacb7693f15594c4ffd50985f6c32db919"}
Jul 07 14:06:27.107956 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:27.107581    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp" event={"ID":"1e402935-ca7d-4a1d-8d70-d244de61ca19","Type":"ContainerStarted","Data":"60eb5b4026b7cdfd8cacc0cd655e16306f89efa5f2fd28a511d879c1040dc81e"}
Jul 07 14:06:27.127650 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:27.127606    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp" podStartSLOduration=2.127589639 podStartE2EDuration="2.127589639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:06:27.126499488 +0000 UTC m=+2321.659405240" watchObservedRunningTime="2026-07-07 14:06:27.127589639 +0000 UTC m=+2321.660495364"
Jul 07 14:06:29.747347 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:29.747311    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:06:36.243745 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:36.243705    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:36.243745 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:36.243756    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:36.257412 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:36.257384    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:37.165101 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:37.165076    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:06:49.342353 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:06:49.342317    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:07:05.291363 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:05.291330    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"]
Jul 07 14:07:05.315267 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:05.315238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:07:06.131920 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:06.131889    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"]
Jul 07 14:07:06.156144 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:06.156112    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:07:10.213499 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.213465    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"]
Jul 07 14:07:10.217015 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.216999    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.219904 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.219885    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b6630e\"/\"rate-test-model-b6630e-kserve-self-signed-certs\""
Jul 07 14:07:10.230254 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.230233    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"]
Jul 07 14:07:10.238439 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.238417    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:07:10.280988 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.280962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-home\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.280988 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.280990    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjg55\" (UniqueName: \"kubernetes.io/projected/b0c462fd-fb98-4a9a-9683-b1736d08f184-kube-api-access-qjg55\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.281158 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.281017    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-dshm\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.281158 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.281101    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-model-cache\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.281158 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.281120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-tmp-dir\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.281262 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.281165    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0c462fd-fb98-4a9a-9683-b1736d08f184-tls-certs\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.381593 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.381557    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-model-cache\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.381809 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.381603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-tmp-dir\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.381809 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.381643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0c462fd-fb98-4a9a-9683-b1736d08f184-tls-certs\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.381809 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.381664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-home\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.381809 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.381681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjg55\" (UniqueName: \"kubernetes.io/projected/b0c462fd-fb98-4a9a-9683-b1736d08f184-kube-api-access-qjg55\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.381809 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.381714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-dshm\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.382082 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.382047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-model-cache\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.382082 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.382075    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-tmp-dir\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.382184 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.382117    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-home\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.383941 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.383922    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-dshm\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.384113 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.384093    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0c462fd-fb98-4a9a-9683-b1736d08f184-tls-certs\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.393372 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.393349    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjg55\" (UniqueName: \"kubernetes.io/projected/b0c462fd-fb98-4a9a-9683-b1736d08f184-kube-api-access-qjg55\") pod \"rate-test-model-b6630e-kserve-5549b7c647-v2mk4\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") " pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.525939 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.525847    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:10.647319 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.647288    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:07:10.663580 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:10.663544    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"]
Jul 07 14:07:10.666354 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:07:10.666326    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0c462fd_fb98_4a9a_9683_b1736d08f184.slice/crio-90d9aa44b150a31e2b53929fefe355382a50b9ee19ffddd7f3eedfea98510a29 WatchSource:0}: Error finding container 90d9aa44b150a31e2b53929fefe355382a50b9ee19ffddd7f3eedfea98510a29: Status 404 returned error can't find the container with id 90d9aa44b150a31e2b53929fefe355382a50b9ee19ffddd7f3eedfea98510a29
Jul 07 14:07:11.314468 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:11.314431    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4" event={"ID":"b0c462fd-fb98-4a9a-9683-b1736d08f184","Type":"ContainerStarted","Data":"71de8b530cb2216d9e28702fd371292e128d3690ca857ff984b56fddfc405cb2"}
Jul 07 14:07:11.314855 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:11.314479    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4" event={"ID":"b0c462fd-fb98-4a9a-9683-b1736d08f184","Type":"ContainerStarted","Data":"90d9aa44b150a31e2b53929fefe355382a50b9ee19ffddd7f3eedfea98510a29"}
Jul 07 14:07:11.337019 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:11.336972    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4" podStartSLOduration=1.336957914 podStartE2EDuration="1.336957914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:07:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:07:11.335703484 +0000 UTC m=+2365.868609209" watchObservedRunningTime="2026-07-07 14:07:11.336957914 +0000 UTC m=+2365.869863638"
Jul 07 14:07:20.311359 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.311298    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc" podUID="3d9331c1-468b-46a9-898d-fbe2b4cb41ac" containerName="main" containerID="cri-o://209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b" gracePeriod=45
Jul 07 14:07:20.526365 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.526334    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:20.526508 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.526383    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:20.539387 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.539364    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:20.558726 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.558698    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:07:20.681582 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681547    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbkmp\" (UniqueName: \"kubernetes.io/projected/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-kube-api-access-xbkmp\") pod \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") "
Jul 07 14:07:20.681850 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681601    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-dshm\") pod \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") "
Jul 07 14:07:20.681850 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681618    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tls-certs\") pod \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") "
Jul 07 14:07:20.681850 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-model-cache\") pod \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") "
Jul 07 14:07:20.681850 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681664    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tmp-dir\") pod \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") "
Jul 07 14:07:20.681850 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681757    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-home\") pod \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\" (UID: \"3d9331c1-468b-46a9-898d-fbe2b4cb41ac\") "
Jul 07 14:07:20.682143 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.681994    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d9331c1-468b-46a9-898d-fbe2b4cb41ac" (UID: "3d9331c1-468b-46a9-898d-fbe2b4cb41ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:20.682206 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.682168    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-home" (OuterVolumeSpecName: "home") pod "3d9331c1-468b-46a9-898d-fbe2b4cb41ac" (UID: "3d9331c1-468b-46a9-898d-fbe2b4cb41ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:20.682274 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.682244    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d9331c1-468b-46a9-898d-fbe2b4cb41ac" (UID: "3d9331c1-468b-46a9-898d-fbe2b4cb41ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:20.683867 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.683848    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-kube-api-access-xbkmp" (OuterVolumeSpecName: "kube-api-access-xbkmp") pod "3d9331c1-468b-46a9-898d-fbe2b4cb41ac" (UID: "3d9331c1-468b-46a9-898d-fbe2b4cb41ac"). InnerVolumeSpecName "kube-api-access-xbkmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:07:20.683927 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.683883    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-dshm" (OuterVolumeSpecName: "dshm") pod "3d9331c1-468b-46a9-898d-fbe2b4cb41ac" (UID: "3d9331c1-468b-46a9-898d-fbe2b4cb41ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:20.683927 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.683917    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d9331c1-468b-46a9-898d-fbe2b4cb41ac" (UID: "3d9331c1-468b-46a9-898d-fbe2b4cb41ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:07:20.783239 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.783203    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:20.783239 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.783232    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbkmp\" (UniqueName: \"kubernetes.io/projected/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-kube-api-access-xbkmp\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:20.783239 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.783245    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:20.783470 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.783257    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:20.783470 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.783268    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:20.783470 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:20.783279    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9331c1-468b-46a9-898d-fbe2b4cb41ac-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:21.152130 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.152077    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp" podUID="1e402935-ca7d-4a1d-8d70-d244de61ca19" containerName="main" containerID="cri-o://531fdabce81f546b3ffb8dd9b21a68bacb7693f15594c4ffd50985f6c32db919" gracePeriod=45
Jul 07 14:07:21.360297 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.360261    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e402935-ca7d-4a1d-8d70-d244de61ca19" containerID="531fdabce81f546b3ffb8dd9b21a68bacb7693f15594c4ffd50985f6c32db919" exitCode=0
Jul 07 14:07:21.360668 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.360330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp" event={"ID":"1e402935-ca7d-4a1d-8d70-d244de61ca19","Type":"ContainerDied","Data":"531fdabce81f546b3ffb8dd9b21a68bacb7693f15594c4ffd50985f6c32db919"}
Jul 07 14:07:21.361616 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.361594    2582 generic.go:356] "Generic (PLEG): container finished" podID="3d9331c1-468b-46a9-898d-fbe2b4cb41ac" containerID="209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b" exitCode=0
Jul 07 14:07:21.361727 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.361633    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"
Jul 07 14:07:21.361727 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.361672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc" event={"ID":"3d9331c1-468b-46a9-898d-fbe2b4cb41ac","Type":"ContainerDied","Data":"209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b"}
Jul 07 14:07:21.361727 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.361701    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc" event={"ID":"3d9331c1-468b-46a9-898d-fbe2b4cb41ac","Type":"ContainerDied","Data":"73899992b8c71ce3f6977c0b5ee347faf4e028f764852bc0202ddad3f9b41091"}
Jul 07 14:07:21.361727 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.361717    2582 scope.go:117] "RemoveContainer" containerID="209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b"
Jul 07 14:07:21.371167 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.371153    2582 scope.go:117] "RemoveContainer" containerID="209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b"
Jul 07 14:07:21.371436 ip-10-0-129-217 kubenswrapper[2582]: E0707 14:07:21.371414    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b\": container with ID starting with 209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b not found: ID does not exist" containerID="209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b"
Jul 07 14:07:21.371522 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.371445    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b"} err="failed to get container status \"209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b\": rpc error: code = NotFound desc = could not find container \"209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b\": container with ID starting with 209d9aa161576c5bcede7451f1c072de9bd7d00db9847da8f2052ed32c6de13b not found: ID does not exist"
Jul 07 14:07:21.375393 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.375373    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:07:21.394070 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.394050    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"]
Jul 07 14:07:21.397652 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.397632    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/sub-test-model-b6630e-kserve-69d7957b7-zn8gc"]
Jul 07 14:07:21.410981 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.410929    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:07:21.490307 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490271    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-tmp-dir\") pod \"1e402935-ca7d-4a1d-8d70-d244de61ca19\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") "
Jul 07 14:07:21.490516 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490333    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-home\") pod \"1e402935-ca7d-4a1d-8d70-d244de61ca19\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") "
Jul 07 14:07:21.490516 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490390    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2l5k\" (UniqueName: \"kubernetes.io/projected/1e402935-ca7d-4a1d-8d70-d244de61ca19-kube-api-access-r2l5k\") pod \"1e402935-ca7d-4a1d-8d70-d244de61ca19\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") "
Jul 07 14:07:21.490516 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490422    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-dshm\") pod \"1e402935-ca7d-4a1d-8d70-d244de61ca19\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") "
Jul 07 14:07:21.490516 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490461    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e402935-ca7d-4a1d-8d70-d244de61ca19-tls-certs\") pod \"1e402935-ca7d-4a1d-8d70-d244de61ca19\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") "
Jul 07 14:07:21.490678 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490555    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1e402935-ca7d-4a1d-8d70-d244de61ca19" (UID: "1e402935-ca7d-4a1d-8d70-d244de61ca19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:21.490678 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490564    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-model-cache\") pod \"1e402935-ca7d-4a1d-8d70-d244de61ca19\" (UID: \"1e402935-ca7d-4a1d-8d70-d244de61ca19\") "
Jul 07 14:07:21.490825 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490801    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-model-cache" (OuterVolumeSpecName: "model-cache") pod "1e402935-ca7d-4a1d-8d70-d244de61ca19" (UID: "1e402935-ca7d-4a1d-8d70-d244de61ca19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:21.490940 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490911    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:21.490940 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.490933    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:21.491306 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.491262    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-home" (OuterVolumeSpecName: "home") pod "1e402935-ca7d-4a1d-8d70-d244de61ca19" (UID: "1e402935-ca7d-4a1d-8d70-d244de61ca19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:21.492920 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.492885    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e402935-ca7d-4a1d-8d70-d244de61ca19-kube-api-access-r2l5k" (OuterVolumeSpecName: "kube-api-access-r2l5k") pod "1e402935-ca7d-4a1d-8d70-d244de61ca19" (UID: "1e402935-ca7d-4a1d-8d70-d244de61ca19"). InnerVolumeSpecName "kube-api-access-r2l5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:07:21.493033 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.492924    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-dshm" (OuterVolumeSpecName: "dshm") pod "1e402935-ca7d-4a1d-8d70-d244de61ca19" (UID: "1e402935-ca7d-4a1d-8d70-d244de61ca19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:07:21.493472 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.493457    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e402935-ca7d-4a1d-8d70-d244de61ca19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1e402935-ca7d-4a1d-8d70-d244de61ca19" (UID: "1e402935-ca7d-4a1d-8d70-d244de61ca19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:07:21.592110 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.592077    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:21.592110 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.592105    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2l5k\" (UniqueName: \"kubernetes.io/projected/1e402935-ca7d-4a1d-8d70-d244de61ca19-kube-api-access-r2l5k\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:21.592110 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.592114    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e402935-ca7d-4a1d-8d70-d244de61ca19-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:21.592330 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:21.592124    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e402935-ca7d-4a1d-8d70-d244de61ca19-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:07:22.078661 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:22.078631    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d9331c1-468b-46a9-898d-fbe2b4cb41ac" path="/var/lib/kubelet/pods/3d9331c1-468b-46a9-898d-fbe2b4cb41ac/volumes"
Jul 07 14:07:22.367966 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:22.367931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp" event={"ID":"1e402935-ca7d-4a1d-8d70-d244de61ca19","Type":"ContainerDied","Data":"60eb5b4026b7cdfd8cacc0cd655e16306f89efa5f2fd28a511d879c1040dc81e"}
Jul 07 14:07:22.368422 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:22.367975    2582 scope.go:117] "RemoveContainer" containerID="531fdabce81f546b3ffb8dd9b21a68bacb7693f15594c4ffd50985f6c32db919"
Jul 07 14:07:22.368422 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:22.367945    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"
Jul 07 14:07:22.384452 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:22.384427    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"]
Jul 07 14:07:22.386821 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:22.386799    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/sub-test-model-f0b1df-kserve-6d9dfbc645-zbgsp"]
Jul 07 14:07:24.078086 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:24.078055    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e402935-ca7d-4a1d-8d70-d244de61ca19" path="/var/lib/kubelet/pods/1e402935-ca7d-4a1d-8d70-d244de61ca19/volumes"
Jul 07 14:07:24.941067 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:24.941031    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:07:34.764725 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:34.764685    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:07:34.863437 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:34.863405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:07:46.831831 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.831769    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"]
Jul 07 14:07:46.832337 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.832288    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e402935-ca7d-4a1d-8d70-d244de61ca19" containerName="main"
Jul 07 14:07:46.832337 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.832307    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e402935-ca7d-4a1d-8d70-d244de61ca19" containerName="main"
Jul 07 14:07:46.832337 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.832326    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d9331c1-468b-46a9-898d-fbe2b4cb41ac" containerName="main"
Jul 07 14:07:46.832337 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.832335    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d9331c1-468b-46a9-898d-fbe2b4cb41ac" containerName="main"
Jul 07 14:07:46.832540 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.832437    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e402935-ca7d-4a1d-8d70-d244de61ca19" containerName="main"
Jul 07 14:07:46.832540 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.832456    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d9331c1-468b-46a9-898d-fbe2b4cb41ac" containerName="main"
Jul 07 14:07:46.837104 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.837083    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:46.840199 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.840180    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"kube-root-ca.crt\""
Jul 07 14:07:46.841558 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.841540    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"openshift-service-ca.crt\""
Jul 07 14:07:46.841675 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.841587    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"default-dockercfg-lbv8g\""
Jul 07 14:07:46.841858 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.841843    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f0b1df\"/\"rate-test-model-f0b1df-kserve-self-signed-certs\""
Jul 07 14:07:46.847463 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:46.847444    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"]
Jul 07 14:07:47.015664 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.015631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-home\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.015664 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.015667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-model-cache\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.015890 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.015685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lchbf\" (UniqueName: \"kubernetes.io/projected/824a6c97-c625-49b1-8a61-979445d86a59-kube-api-access-lchbf\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.015890 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.015703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-dshm\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.015890 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.015725    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-tmp-dir\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.015890 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.015788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824a6c97-c625-49b1-8a61-979445d86a59-tls-certs\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.116633 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.116603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-home\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.116843 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.116639    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-model-cache\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.116843 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.116658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lchbf\" (UniqueName: \"kubernetes.io/projected/824a6c97-c625-49b1-8a61-979445d86a59-kube-api-access-lchbf\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.116843 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.116687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-dshm\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.116843 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.116709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-tmp-dir\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.117085 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.116840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824a6c97-c625-49b1-8a61-979445d86a59-tls-certs\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.117085 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.117056    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-home\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.117172 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.117123    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-model-cache\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.117172 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.117158    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-tmp-dir\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.119043 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.119016    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-dshm\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.119178 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.119161    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824a6c97-c625-49b1-8a61-979445d86a59-tls-certs\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.128344 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.128324    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lchbf\" (UniqueName: \"kubernetes.io/projected/824a6c97-c625-49b1-8a61-979445d86a59-kube-api-access-lchbf\") pod \"rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") " pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.146382 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.146352    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:47.278485 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.278440    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"]
Jul 07 14:07:47.280724 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:07:47.280699    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod824a6c97_c625_49b1_8a61_979445d86a59.slice/crio-be632508552d5df4150604f3c98ee99cc744ecd45350f9eddc5d089bc43c5525 WatchSource:0}: Error finding container be632508552d5df4150604f3c98ee99cc744ecd45350f9eddc5d089bc43c5525: Status 404 returned error can't find the container with id be632508552d5df4150604f3c98ee99cc744ecd45350f9eddc5d089bc43c5525
Jul 07 14:07:47.479712 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.479618    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq" event={"ID":"824a6c97-c625-49b1-8a61-979445d86a59","Type":"ContainerStarted","Data":"d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a"}
Jul 07 14:07:47.479712 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.479658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq" event={"ID":"824a6c97-c625-49b1-8a61-979445d86a59","Type":"ContainerStarted","Data":"be632508552d5df4150604f3c98ee99cc744ecd45350f9eddc5d089bc43c5525"}
Jul 07 14:07:47.498207 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.498155    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq" podStartSLOduration=1.498139079 podStartE2EDuration="1.498139079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:07:47.496320336 +0000 UTC m=+2402.029226086" watchObservedRunningTime="2026-07-07 14:07:47.498139079 +0000 UTC m=+2402.031044805"
Jul 07 14:07:47.558041 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:47.558000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:07:51.039160 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:51.039124    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:07:57.146834 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:57.146798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:57.147223 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:57.146850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:57.159358 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:57.159328    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:07:57.542079 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:07:57.542002    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:08:11.743703 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:11.743663    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:08:25.239445 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:25.239408    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:08:25.564146 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:25.564068    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:25.670794 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:25.670744    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:30.534178 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:30.534146    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"]
Jul 07 14:08:30.557554 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:30.557521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:31.406137 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:31.406105    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"]
Jul 07 14:08:31.552946 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:31.552914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:32.487714 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:32.487669    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:32.824799 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:32.824699    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:32.922570 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:32.922542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:34.662502 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:34.662463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:34.799984 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:34.799952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:37.958582 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:37.958545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:41.632479 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:41.632446    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:08:45.554148 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.554101    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4" podUID="b0c462fd-fb98-4a9a-9683-b1736d08f184" containerName="main" containerID="cri-o://71de8b530cb2216d9e28702fd371292e128d3690ca857ff984b56fddfc405cb2" gracePeriod=45
Jul 07 14:08:45.745265 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.745231    2582 generic.go:356] "Generic (PLEG): container finished" podID="b0c462fd-fb98-4a9a-9683-b1736d08f184" containerID="71de8b530cb2216d9e28702fd371292e128d3690ca857ff984b56fddfc405cb2" exitCode=0
Jul 07 14:08:45.745428 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.745301    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4" event={"ID":"b0c462fd-fb98-4a9a-9683-b1736d08f184","Type":"ContainerDied","Data":"71de8b530cb2216d9e28702fd371292e128d3690ca857ff984b56fddfc405cb2"}
Jul 07 14:08:45.795614 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.795588    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:08:45.869562 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869532    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0c462fd-fb98-4a9a-9683-b1736d08f184-tls-certs\") pod \"b0c462fd-fb98-4a9a-9683-b1736d08f184\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") "
Jul 07 14:08:45.869724 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869581    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-tmp-dir\") pod \"b0c462fd-fb98-4a9a-9683-b1736d08f184\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") "
Jul 07 14:08:45.869724 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869657    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-home\") pod \"b0c462fd-fb98-4a9a-9683-b1736d08f184\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") "
Jul 07 14:08:45.869882 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869729    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjg55\" (UniqueName: \"kubernetes.io/projected/b0c462fd-fb98-4a9a-9683-b1736d08f184-kube-api-access-qjg55\") pod \"b0c462fd-fb98-4a9a-9683-b1736d08f184\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") "
Jul 07 14:08:45.869882 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869763    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-dshm\") pod \"b0c462fd-fb98-4a9a-9683-b1736d08f184\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") "
Jul 07 14:08:45.869882 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869825    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-model-cache\") pod \"b0c462fd-fb98-4a9a-9683-b1736d08f184\" (UID: \"b0c462fd-fb98-4a9a-9683-b1736d08f184\") "
Jul 07 14:08:45.870019 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869892    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0c462fd-fb98-4a9a-9683-b1736d08f184" (UID: "b0c462fd-fb98-4a9a-9683-b1736d08f184"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:45.870019 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.869967    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-home" (OuterVolumeSpecName: "home") pod "b0c462fd-fb98-4a9a-9683-b1736d08f184" (UID: "b0c462fd-fb98-4a9a-9683-b1736d08f184"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:45.870168 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.870139    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0c462fd-fb98-4a9a-9683-b1736d08f184" (UID: "b0c462fd-fb98-4a9a-9683-b1736d08f184"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:45.870240 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.870175    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:45.870240 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.870191    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:45.871740 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.871709    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-dshm" (OuterVolumeSpecName: "dshm") pod "b0c462fd-fb98-4a9a-9683-b1736d08f184" (UID: "b0c462fd-fb98-4a9a-9683-b1736d08f184"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:45.871740 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.871718    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0c462fd-fb98-4a9a-9683-b1736d08f184-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0c462fd-fb98-4a9a-9683-b1736d08f184" (UID: "b0c462fd-fb98-4a9a-9683-b1736d08f184"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:08:45.871917 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.871749    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0c462fd-fb98-4a9a-9683-b1736d08f184-kube-api-access-qjg55" (OuterVolumeSpecName: "kube-api-access-qjg55") pod "b0c462fd-fb98-4a9a-9683-b1736d08f184" (UID: "b0c462fd-fb98-4a9a-9683-b1736d08f184"). InnerVolumeSpecName "kube-api-access-qjg55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:08:45.970906 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.970879    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:45.970906 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.970901    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0c462fd-fb98-4a9a-9683-b1736d08f184-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:45.970906 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.970910    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0c462fd-fb98-4a9a-9683-b1736d08f184-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:45.971132 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:45.970918    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjg55\" (UniqueName: \"kubernetes.io/projected/b0c462fd-fb98-4a9a-9683-b1736d08f184-kube-api-access-qjg55\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:46.425848 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.425761    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq" podUID="824a6c97-c625-49b1-8a61-979445d86a59" containerName="main" containerID="cri-o://d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a" gracePeriod=45
Jul 07 14:08:46.677737 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.677678    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:08:46.753472 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.753440    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4" event={"ID":"b0c462fd-fb98-4a9a-9683-b1736d08f184","Type":"ContainerDied","Data":"90d9aa44b150a31e2b53929fefe355382a50b9ee19ffddd7f3eedfea98510a29"}
Jul 07 14:08:46.753652 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.753446    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"
Jul 07 14:08:46.753652 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.753485    2582 scope.go:117] "RemoveContainer" containerID="71de8b530cb2216d9e28702fd371292e128d3690ca857ff984b56fddfc405cb2"
Jul 07 14:08:46.754932 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.754909    2582 generic.go:356] "Generic (PLEG): container finished" podID="824a6c97-c625-49b1-8a61-979445d86a59" containerID="d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a" exitCode=0
Jul 07 14:08:46.755074 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.754955    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"
Jul 07 14:08:46.755074 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.754967    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq" event={"ID":"824a6c97-c625-49b1-8a61-979445d86a59","Type":"ContainerDied","Data":"d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a"}
Jul 07 14:08:46.755074 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.754993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq" event={"ID":"824a6c97-c625-49b1-8a61-979445d86a59","Type":"ContainerDied","Data":"be632508552d5df4150604f3c98ee99cc744ecd45350f9eddc5d089bc43c5525"}
Jul 07 14:08:46.761896 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.761881    2582 scope.go:117] "RemoveContainer" containerID="d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a"
Jul 07 14:08:46.769478 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.769457    2582 scope.go:117] "RemoveContainer" containerID="d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a"
Jul 07 14:08:46.769737 ip-10-0-129-217 kubenswrapper[2582]: E0707 14:08:46.769714    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a\": container with ID starting with d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a not found: ID does not exist" containerID="d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a"
Jul 07 14:08:46.769837 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.769749    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a"} err="failed to get container status \"d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a\": rpc error: code = NotFound desc = could not find container \"d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a\": container with ID starting with d6da018249206c0351ea88fe17da67b66487b62521cfc5f3d32c4601fa85f93a not found: ID does not exist"
Jul 07 14:08:46.771028 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.771011    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"]
Jul 07 14:08:46.774533 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.774516    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b6630e/rate-test-model-b6630e-kserve-5549b7c647-v2mk4"]
Jul 07 14:08:46.778915 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.778898    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lchbf\" (UniqueName: \"kubernetes.io/projected/824a6c97-c625-49b1-8a61-979445d86a59-kube-api-access-lchbf\") pod \"824a6c97-c625-49b1-8a61-979445d86a59\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") "
Jul 07 14:08:46.779016 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.778924    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824a6c97-c625-49b1-8a61-979445d86a59-tls-certs\") pod \"824a6c97-c625-49b1-8a61-979445d86a59\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") "
Jul 07 14:08:46.779016 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.778969    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-dshm\") pod \"824a6c97-c625-49b1-8a61-979445d86a59\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") "
Jul 07 14:08:46.779097 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779040    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-home\") pod \"824a6c97-c625-49b1-8a61-979445d86a59\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") "
Jul 07 14:08:46.779133 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779093    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-model-cache\") pod \"824a6c97-c625-49b1-8a61-979445d86a59\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") "
Jul 07 14:08:46.779176 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-tmp-dir\") pod \"824a6c97-c625-49b1-8a61-979445d86a59\" (UID: \"824a6c97-c625-49b1-8a61-979445d86a59\") "
Jul 07 14:08:46.779304 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779282    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-home" (OuterVolumeSpecName: "home") pod "824a6c97-c625-49b1-8a61-979445d86a59" (UID: "824a6c97-c625-49b1-8a61-979445d86a59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:46.779388 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779334    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-model-cache" (OuterVolumeSpecName: "model-cache") pod "824a6c97-c625-49b1-8a61-979445d86a59" (UID: "824a6c97-c625-49b1-8a61-979445d86a59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:46.779451 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779430    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "824a6c97-c625-49b1-8a61-979445d86a59" (UID: "824a6c97-c625-49b1-8a61-979445d86a59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:46.779532 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779508    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:46.779646 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779537    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:46.779646 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.779552    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:46.781041 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.781020    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824a6c97-c625-49b1-8a61-979445d86a59-kube-api-access-lchbf" (OuterVolumeSpecName: "kube-api-access-lchbf") pod "824a6c97-c625-49b1-8a61-979445d86a59" (UID: "824a6c97-c625-49b1-8a61-979445d86a59"). InnerVolumeSpecName "kube-api-access-lchbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:08:46.781142 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.781054    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/824a6c97-c625-49b1-8a61-979445d86a59-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "824a6c97-c625-49b1-8a61-979445d86a59" (UID: "824a6c97-c625-49b1-8a61-979445d86a59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:08:46.781142 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.781070    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-dshm" (OuterVolumeSpecName: "dshm") pod "824a6c97-c625-49b1-8a61-979445d86a59" (UID: "824a6c97-c625-49b1-8a61-979445d86a59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:08:46.880472 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.880443    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lchbf\" (UniqueName: \"kubernetes.io/projected/824a6c97-c625-49b1-8a61-979445d86a59-kube-api-access-lchbf\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:46.880472 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.880469    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824a6c97-c625-49b1-8a61-979445d86a59-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:46.880472 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:46.880478    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824a6c97-c625-49b1-8a61-979445d86a59-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\""
Jul 07 14:08:47.079239 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:47.079210    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"]
Jul 07 14:08:47.086207 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:47.086184    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f0b1df/rate-test-model-f0b1df-kserve-8cccd99f5-vbzwq"]
Jul 07 14:08:48.077865 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:48.077839    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="824a6c97-c625-49b1-8a61-979445d86a59" path="/var/lib/kubelet/pods/824a6c97-c625-49b1-8a61-979445d86a59/volumes"
Jul 07 14:08:48.078228 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:48.078206    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0c462fd-fb98-4a9a-9683-b1736d08f184" path="/var/lib/kubelet/pods/b0c462fd-fb98-4a9a-9683-b1736d08f184/volumes"
Jul 07 14:08:57.142041 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:57.142003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:57.417170 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:57.417079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:57.655994 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:57.655956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:57.926548 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:57.926515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:58.265207 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:58.265131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:08:58.335461 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:08:58.335425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:00.733952 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:00.733920    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r4jvr"]
Jul 07 14:09:02.782753 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:02.782719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:02.886068 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:02.886030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:04.511437 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:04.511394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:04.584584 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:04.584547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:04.824263 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:04.824180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:05.046969 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:05.046937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:05.461473 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:05.461441    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69ccf8b4bd-44dsz_d9a2ed53-2cb5-40b3-b74f-a1063302c665/authorino/0.log"
Jul 07 14:09:11.138178 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:11.138144    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:11.211892 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:11.211865    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f9489749-767tj_b6295a72-6bd0-401e-a6f6-5790bf4c3fdc/copy-manifests/0.log"
Jul 07 14:09:11.221292 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:11.221267    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f9489749-767tj_b6295a72-6bd0-401e-a6f6-5790bf4c3fdc/manager/0.log"
Jul 07 14:09:12.915769 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:12.915742    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69ccf8b4bd-44dsz_d9a2ed53-2cb5-40b3-b74f-a1063302c665/authorino/0.log"
Jul 07 14:09:13.032867 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:13.032837    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-b6ntf_744040c2-d02e-4212-acb6-48e0d59bfe0c/manager/0.log"
Jul 07 14:09:13.625388 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:13.625354    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-r4jvr_65c1a5ef-0724-425c-9728-a660a386ad43/limitador/0.log"
Jul 07 14:09:14.467107 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:14.467075    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-87699d5c5-zxqbx_af2337ea-212c-4499-9d55-35f9a0b95e3d/kube-auth-proxy/0.log"
Jul 07 14:09:15.872432 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:15.872407    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-hdcfc_5b01bd0d-31d7-4eb2-bb8a-e7d7db75db48/main/0.log"
Jul 07 14:09:19.751986 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.751951    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kxg6n/must-gather-h8cqs"]
Jul 07 14:09:19.752464 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.752425    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0c462fd-fb98-4a9a-9683-b1736d08f184" containerName="main"
Jul 07 14:09:19.752464 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.752443    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0c462fd-fb98-4a9a-9683-b1736d08f184" containerName="main"
Jul 07 14:09:19.752584 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.752465    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824a6c97-c625-49b1-8a61-979445d86a59" containerName="main"
Jul 07 14:09:19.752584 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.752475    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="824a6c97-c625-49b1-8a61-979445d86a59" containerName="main"
Jul 07 14:09:19.752584 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.752569    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0c462fd-fb98-4a9a-9683-b1736d08f184" containerName="main"
Jul 07 14:09:19.752743 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.752587    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="824a6c97-c625-49b1-8a61-979445d86a59" containerName="main"
Jul 07 14:09:19.755695 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.755675    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:19.758612 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.758592    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kxg6n\"/\"kube-root-ca.crt\""
Jul 07 14:09:19.758742 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.758612    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kxg6n\"/\"openshift-service-ca.crt\""
Jul 07 14:09:19.769091 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.769068    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxg6n/must-gather-h8cqs"]
Jul 07 14:09:19.774493 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.774461    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:19.879380 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.879346    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4a7009d8-08fb-4858-bf09-76b4288bfaf1-must-gather-output\") pod \"must-gather-h8cqs\" (UID: \"4a7009d8-08fb-4858-bf09-76b4288bfaf1\") " pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:19.879560 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.879427    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mc4z\" (UniqueName: \"kubernetes.io/projected/4a7009d8-08fb-4858-bf09-76b4288bfaf1-kube-api-access-7mc4z\") pod \"must-gather-h8cqs\" (UID: \"4a7009d8-08fb-4858-bf09-76b4288bfaf1\") " pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:19.927115 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.927070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log" limit=1024
Jul 07 14:09:19.980451 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.980418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mc4z\" (UniqueName: \"kubernetes.io/projected/4a7009d8-08fb-4858-bf09-76b4288bfaf1-kube-api-access-7mc4z\") pod \"must-gather-h8cqs\" (UID: \"4a7009d8-08fb-4858-bf09-76b4288bfaf1\") " pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:19.980577 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.980479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4a7009d8-08fb-4858-bf09-76b4288bfaf1-must-gather-output\") pod \"must-gather-h8cqs\" (UID: \"4a7009d8-08fb-4858-bf09-76b4288bfaf1\") " pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:19.980828 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.980810    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4a7009d8-08fb-4858-bf09-76b4288bfaf1-must-gather-output\") pod \"must-gather-h8cqs\" (UID: \"4a7009d8-08fb-4858-bf09-76b4288bfaf1\") " pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:19.997876 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:19.997854    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mc4z\" (UniqueName: \"kubernetes.io/projected/4a7009d8-08fb-4858-bf09-76b4288bfaf1-kube-api-access-7mc4z\") pod \"must-gather-h8cqs\" (UID: \"4a7009d8-08fb-4858-bf09-76b4288bfaf1\") " pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:20.065021 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:20.064927    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxg6n/must-gather-h8cqs"
Jul 07 14:09:20.197952 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:20.197877    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxg6n/must-gather-h8cqs"]
Jul 07 14:09:20.200161 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:09:20.200136    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a7009d8_08fb_4858_bf09_76b4288bfaf1.slice/crio-e475b3bc7bea2ee89e597418cff6b76da0b8a49faaeb8ec8e64fc1505f4d5557 WatchSource:0}: Error finding container e475b3bc7bea2ee89e597418cff6b76da0b8a49faaeb8ec8e64fc1505f4d5557: Status 404 returned error can't find the container with id e475b3bc7bea2ee89e597418cff6b76da0b8a49faaeb8ec8e64fc1505f4d5557
Jul 07 14:09:20.201878 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:20.201865    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:09:20.919905 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:20.919814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxg6n/must-gather-h8cqs" event={"ID":"4a7009d8-08fb-4858-bf09-76b4288bfaf1","Type":"ContainerStarted","Data":"d5138306c75f1187f821e6699823da51d46bd931a4bf5c409a43000d69257901"}
Jul 07 14:09:20.919905 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:20.919869    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxg6n/must-gather-h8cqs" event={"ID":"4a7009d8-08fb-4858-bf09-76b4288bfaf1","Type":"ContainerStarted","Data":"e475b3bc7bea2ee89e597418cff6b76da0b8a49faaeb8ec8e64fc1505f4d5557"}
Jul 07 14:09:21.928794 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:21.928673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxg6n/must-gather-h8cqs" event={"ID":"4a7009d8-08fb-4858-bf09-76b4288bfaf1","Type":"ContainerStarted","Data":"f8fd975eb52b5702cfd7e8a9e4de8bd50e708034991433872df48971f056a2d8"}
Jul 07 14:09:21.948192 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:21.948129    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kxg6n/must-gather-h8cqs" podStartSLOduration=2.360827777 podStartE2EDuration="2.948106973s" totalImagesPullingTime="587.279196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:21.947723822 +0000 UTC m=+2496.480629548" watchObservedRunningTime="2026-07-07 14:09:21.948106973 +0000 UTC m=+2496.481012697"
Jul 07 14:09:22.607750 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:22.607725    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pjn5s_f8332404-cb66-4442-a5e4-59b4ea994ab7/global-pull-secret-syncer/0.log"
Jul 07 14:09:22.663138 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:22.663107    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-48bvw_c112cf11-8255-4826-906e-0b8a815f125b/konnectivity-agent/0.log"
Jul 07 14:09:22.752939 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:22.752909    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-217.ec2.internal_1c857f80ddb13de0ede08c410a7a78ba/haproxy/0.log"
Jul 07 14:09:27.120399 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:27.119875    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69ccf8b4bd-44dsz_d9a2ed53-2cb5-40b3-b74f-a1063302c665/authorino/0.log"
Jul 07 14:09:27.151929 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:27.151899    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-b6ntf_744040c2-d02e-4212-acb6-48e0d59bfe0c/manager/0.log"
Jul 07 14:09:27.700159 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:27.700129    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-r4jvr_65c1a5ef-0724-425c-9728-a660a386ad43/limitador/0.log"
Jul 07 14:09:29.765404 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:29.765368    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8rrlx_928a877b-4cbb-4a66-b732-9309e90cfd0e/kube-state-metrics/0.log"
Jul 07 14:09:29.786292 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:29.786210    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8rrlx_928a877b-4cbb-4a66-b732-9309e90cfd0e/kube-rbac-proxy-main/0.log"
Jul 07 14:09:29.812405 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:29.812290    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8rrlx_928a877b-4cbb-4a66-b732-9309e90cfd0e/kube-rbac-proxy-self/0.log"
Jul 07 14:09:29.859018 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:29.858989    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-796f687785-gfk4v_99c88269-9e09-47c2-a89e-0f87a136de78/metrics-server/0.log"
Jul 07 14:09:29.941246 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:29.941218    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-j6xqr_05f8c282-030f-4b3a-bbf3-58cbadb98560/monitoring-plugin/0.log"
Jul 07 14:09:30.004030 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.004002    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-275z7_b398a611-22c3-4ec9-a0bb-5395e6726d66/node-exporter/0.log"
Jul 07 14:09:30.059087 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.059019    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-275z7_b398a611-22c3-4ec9-a0bb-5395e6726d66/kube-rbac-proxy/0.log"
Jul 07 14:09:30.097619 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.097598    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-275z7_b398a611-22c3-4ec9-a0bb-5395e6726d66/init-textfile/0.log"
Jul 07 14:09:30.328747 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.328717    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-mthk7_77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7/kube-rbac-proxy-main/0.log"
Jul 07 14:09:30.367806 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.367710    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-mthk7_77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7/kube-rbac-proxy-self/0.log"
Jul 07 14:09:30.409458 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.409410    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-mthk7_77825edc-aeb8-44aa-8fd9-d0fd69c3dcb7/openshift-state-metrics/0.log"
Jul 07 14:09:30.453635 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.453599    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/prometheus/0.log"
Jul 07 14:09:30.481392 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.481369    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/config-reloader/0.log"
Jul 07 14:09:30.506766 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.506723    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/thanos-sidecar/0.log"
Jul 07 14:09:30.533303 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.533278    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/kube-rbac-proxy-web/0.log"
Jul 07 14:09:30.558302 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.558266    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/kube-rbac-proxy/0.log"
Jul 07 14:09:30.583702 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.583671    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/kube-rbac-proxy-thanos/0.log"
Jul 07 14:09:30.608125 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.608102    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e2d393f-84f8-4ad0-a809-dc7ae03823c3/init-config-reloader/0.log"
Jul 07 14:09:30.641966 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.641872    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-9h48r_ea3c1f24-ec99-42b1-9866-05095d0e406f/prometheus-operator/0.log"
Jul 07 14:09:30.668330 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.668300    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-9h48r_ea3c1f24-ec99-42b1-9866-05095d0e406f/kube-rbac-proxy/0.log"
Jul 07 14:09:30.693673 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:30.693649    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-njnlp_c0ee24ed-745a-4757-b435-00c8ac0d2689/prometheus-operator-admission-webhook/0.log"
Jul 07 14:09:33.289719 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.289685    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"]
Jul 07 14:09:33.297858 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.297829    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.302738 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.302712    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kxg6n\"/\"default-dockercfg-nkdqx\""
Jul 07 14:09:33.306275 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.306200    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"]
Jul 07 14:09:33.424672 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.424630    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-lib-modules\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.424992 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.424970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-podres\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.425134 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.425119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-proc\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.425326 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.425308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wmzd\" (UniqueName: \"kubernetes.io/projected/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-kube-api-access-9wmzd\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.425546 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.425527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-sys\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526787 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-podres\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526970 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-proc\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526970 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmzd\" (UniqueName: \"kubernetes.io/projected/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-kube-api-access-9wmzd\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526970 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-sys\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526970 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526901    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-podres\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526970 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526915    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-proc\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.526970 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.526914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-lib-modules\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.527301 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.527025    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-sys\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.527301 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.527090    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-lib-modules\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.530148 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.530114    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6977954b85-w295v_2ecaddb3-75b2-4683-9881-12b75da88a05/console/0.log"
Jul 07 14:09:33.539067 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.539038    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wmzd\" (UniqueName: \"kubernetes.io/projected/a86b0c31-4fc3-4b95-ba8a-6ffc85824be5-kube-api-access-9wmzd\") pod \"perf-node-gather-daemonset-sv5jl\" (UID: \"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5\") " pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.613141 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.613110    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:33.776834 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:33.776804    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"]
Jul 07 14:09:33.779702 ip-10-0-129-217 kubenswrapper[2582]: W0707 14:09:33.779670    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda86b0c31_4fc3_4b95_ba8a_6ffc85824be5.slice/crio-5f99518faef7cf0281ddd0616656ea71367352908ef4361bce5ee5d33b3ed93b WatchSource:0}: Error finding container 5f99518faef7cf0281ddd0616656ea71367352908ef4361bce5ee5d33b3ed93b: Status 404 returned error can't find the container with id 5f99518faef7cf0281ddd0616656ea71367352908ef4361bce5ee5d33b3ed93b
Jul 07 14:09:34.013227 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:34.013186    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl" event={"ID":"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5","Type":"ContainerStarted","Data":"29d526f877fee9d3979e4c24ed11cc4d662a0e3acdffd1c68d701d7b22814204"}
Jul 07 14:09:34.013425 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:34.013254    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl" event={"ID":"a86b0c31-4fc3-4b95-ba8a-6ffc85824be5","Type":"ContainerStarted","Data":"5f99518faef7cf0281ddd0616656ea71367352908ef4361bce5ee5d33b3ed93b"}
Jul 07 14:09:34.013705 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:34.013681    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:34.032017 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:34.031955    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl" podStartSLOduration=1.03193658 podStartE2EDuration="1.03193658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:34.031617984 +0000 UTC m=+2508.564523709" watchObservedRunningTime="2026-07-07 14:09:34.03193658 +0000 UTC m=+2508.564842309"
Jul 07 14:09:35.232704 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:35.232678    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lgc2_b3f1c729-7dac-4fc7-bd63-780b499660b9/dns/0.log"
Jul 07 14:09:35.255759 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:35.255707    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lgc2_b3f1c729-7dac-4fc7-bd63-780b499660b9/kube-rbac-proxy/0.log"
Jul 07 14:09:35.403544 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:35.403515    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5jqwk_8db6f404-a4ed-47b3-9850-eb6b8b776e74/dns-node-resolver/0.log"
Jul 07 14:09:36.098533 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:36.098502    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhntt_7073646d-a4f6-4a6d-ad97-fe2ad699b39a/node-ca/0.log"
Jul 07 14:09:37.234256 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:37.234210    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-87699d5c5-zxqbx_af2337ea-212c-4499-9d55-35f9a0b95e3d/kube-auth-proxy/0.log"
Jul 07 14:09:38.186716 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:38.186665    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j8n8j_76b41427-1a41-4266-b6ef-6948526ec2be/serve-healthcheck-canary/0.log"
Jul 07 14:09:38.686727 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:38.686683    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hl4gp_d61a68c1-2c75-40a8-9e32-0e903fbd9d13/kube-rbac-proxy/0.log"
Jul 07 14:09:38.709541 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:38.709520    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hl4gp_d61a68c1-2c75-40a8-9e32-0e903fbd9d13/exporter/0.log"
Jul 07 14:09:38.733040 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:38.733012    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hl4gp_d61a68c1-2c75-40a8-9e32-0e903fbd9d13/extractor/0.log"
Jul 07 14:09:40.030132 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:40.029171    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kxg6n/perf-node-gather-daemonset-sv5jl"
Jul 07 14:09:41.507109 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:41.507083    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f9489749-767tj_b6295a72-6bd0-401e-a6f6-5790bf4c3fdc/manager/0.log"
Jul 07 14:09:41.541080 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:41.541042    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f9489749-767tj_b6295a72-6bd0-401e-a6f6-5790bf4c3fdc/copy-manifests/0.log"
Jul 07 14:09:42.987658 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:42.987565    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-87699d5c5-zxqbx_af2337ea-212c-4499-9d55-35f9a0b95e3d/kube-auth-proxy/0.log"
Jul 07 14:09:43.344033 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:43.343947    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5dbf57879c-2gfcs_f7146892-c42b-4032-95ff-44e174195312/manager/0.log"
Jul 07 14:09:44.493519 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:44.493491    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-87699d5c5-zxqbx_af2337ea-212c-4499-9d55-35f9a0b95e3d/kube-auth-proxy/0.log"
Jul 07 14:09:50.300847 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:50.300756    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r4gz_7f9a5066-7b52-4696-b4b1-7cb4f0e25598/kube-multus-additional-cni-plugins/0.log"
Jul 07 14:09:50.341921 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:50.341889    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r4gz_7f9a5066-7b52-4696-b4b1-7cb4f0e25598/egress-router-binary-copy/0.log"
Jul 07 14:09:50.403842 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:50.403819    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r4gz_7f9a5066-7b52-4696-b4b1-7cb4f0e25598/cni-plugins/0.log"
Jul 07 14:09:50.431664 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:50.431637    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r4gz_7f9a5066-7b52-4696-b4b1-7cb4f0e25598/bond-cni-plugin/0.log"
Jul 07 14:09:50.477228 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:50.477200    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r4gz_7f9a5066-7b52-4696-b4b1-7cb4f0e25598/routeoverride-cni/0.log"
Jul 07 14:09:50.502329 ip-10-0-129-217 kubenswrapper[2582]: I0707 14:09:50.502293    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r4gz_7f9a5066-7b52-4696-b4b1-7cb4f0e25598/whereabouts-cni-bincopy/0.log"