Jul 11 07:38:12.574358 ip-10-0-141-181 systemd[1]: Starting Kubernetes Kubelet...
Jul 11 07:38:12.944568 ip-10-0-141-181 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 11 07:38:12.944568 ip-10-0-141-181 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 11 07:38:12.944568 ip-10-0-141-181 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 11 07:38:12.944568 ip-10-0-141-181 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 11 07:38:12.944568 ip-10-0-141-181 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 11 07:38:12.946991 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.946845    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 11 07:38:12.952285 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952257    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 07:38:12.952285 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952277    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 07:38:12.952285 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952282    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 07:38:12.952285 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952286    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 07:38:12.952285 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952290    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952295    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952299    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952303    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952306    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952311    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952315    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952319    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952322    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952325    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952329    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952332    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952336    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952340    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952344    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952350    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952355    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952359    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952363    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 07:38:12.952575 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952366    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952371    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952374    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952378    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952382    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952386    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952398    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952402    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952406    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952410    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952414    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952418    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952421    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952425    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952429    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952433    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952439    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952443    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952447    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952451    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 07:38:12.953363 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952455    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952460    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952464    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952472    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952477    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952483    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952487    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952494    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952498    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952502    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952507    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952511    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952515    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952519    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952523    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952527    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952531    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952536    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952540    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952545    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 07:38:12.954211 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952549    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952553    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952557    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952561    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952566    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952570    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952573    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952579    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952583    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952589    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952594    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952598    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952602    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952606    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952611    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952615    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952619    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952623    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952627    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952631    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 07:38:12.954982 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952637    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952641    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952645    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952649    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952653    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952657    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952661    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952665    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952669    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952673    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952677    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952682    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952686    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952691    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952695    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.952701    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953867    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953878    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953884    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953890    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 07:38:12.955797 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953894    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953899    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953906    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953911    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953916    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953921    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953927    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953931    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953935    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953940    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953944    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953949    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953953    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953957    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953962    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953966    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953971    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953975    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953979    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953984    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 07:38:12.956656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953987    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953992    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.953996    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954000    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954005    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954009    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954013    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954017    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954021    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954026    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954031    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954036    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954040    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954045    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954050    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954054    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954058    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954062    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954066    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 07:38:12.957323 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954070    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954074    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954079    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954083    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954107    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954112    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954116    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954121    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954126    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954130    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954134    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954139    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954143    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954147    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954152    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954156    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954160    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954164    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954168    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954173    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 07:38:12.957848 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954177    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954181    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954185    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954189    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954194    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954198    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954202    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954206    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954210    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954214    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954218    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954223    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954227    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954231    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954236    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954240    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954246    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954252    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954257    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954262    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 07:38:12.958336 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954267    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954272    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954276    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954280    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954284    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954289    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954293    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954298    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954302    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954306    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954310    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954314    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954318    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954322    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954326    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954330    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954932    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954939    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954944    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 07:38:12.958954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954948    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954952    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954956    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954961    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954965    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954970    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954974    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954978    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954982    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954986    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954990    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954993    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.954997    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955002    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955007    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955011    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955015    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955019    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955026    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955032    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 07:38:12.959553 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955038    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955043    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955049    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955054    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955060    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955064    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955069    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955073    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955077    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955081    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955102    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955107    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955112    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955115    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955120    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955125    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955129    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955133    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955137    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955141    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 07:38:12.960014 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955145    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955149    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955152    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955157    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955161    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955165    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955170    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955175    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955179    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955182    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955187    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955191    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955195    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955199    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955203    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955208    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955212    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955216    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955220    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955224    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 07:38:12.960549 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955228    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955232    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955236    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955240    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955244    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955249    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955253    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955257    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955261    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955265    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955269    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955273    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955278    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955282    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955286    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955290    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955295    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955299    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955303    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955308    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 07:38:12.961022 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955314    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955319    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955323    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955327    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955331    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955335    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955339    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955343    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955347    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955351    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955356    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955360    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955369    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955376    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955381    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.955386    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955485    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955496    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955506    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955513    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 11 07:38:12.961513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955519    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955524    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955531    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955537    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955542    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955547    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955552    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955557    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955562    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955567    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955571    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955576    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955581    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955586    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955593    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955597    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955602    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955607    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955612    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955618    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955623    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955627    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955632    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955637    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 11 07:38:12.961970 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955644    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955651    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955656    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955661    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955667    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955672    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955677    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955682    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955686    2572 flags.go:64] FLAG: --enable-server="true"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955691    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955699    2572 flags.go:64] FLAG: --event-burst="100"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955704    2572 flags.go:64] FLAG: --event-qps="50"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955708    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955713    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955718    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955724    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955728    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955733    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955737    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955742    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955747    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955752    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955757    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955762    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955767    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 11 07:38:12.962553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955772    2572 flags.go:64] FLAG: --feature-gates=""
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955777    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955782    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955787    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955791    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955796    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955801    2572 flags.go:64] FLAG: --help="false"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955807    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-141-181.ec2.internal"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955813    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955817    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955822    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955827    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955833    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955838    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955842    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955846    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955851    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955856    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955861    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955865    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955870    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955874    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955879    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955883    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 11 07:38:12.963221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955888    2572 flags.go:64] FLAG: --lock-file=""
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955892    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955897    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955901    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955910    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955915    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955919    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955924    2572 flags.go:64] FLAG: --logging-format="text"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955928    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955933    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955938    2572 flags.go:64] FLAG: --manifest-url=""
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955942    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955949    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955954    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955961    2572 flags.go:64] FLAG: --max-pods="110"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955966    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955972    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955977    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955982    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955987    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955992    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.955996    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956007    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956012    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956016    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 11 07:38:12.963802 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956021    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956026    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956035    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956040    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956045    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956049    2572 flags.go:64] FLAG: --port="10250"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956054    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956058    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07bd637374f71f657"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956063    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956067    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956072    2572 flags.go:64] FLAG: --register-node="true"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956077    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956082    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956104    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956109    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956113    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956119    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956124    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956129    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956133    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956138    2572 flags.go:64] FLAG: --runonce="false"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956142    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956148    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956153    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956161    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 11 07:38:12.964412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956166    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956170    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956175    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956180    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956185    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956189    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956194    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956198    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956203    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956208    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956212    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956221    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956225    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956230    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956236    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956241    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956245    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956250    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956254    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956259    2572 flags.go:64] FLAG: --v="2"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956265    2572 flags.go:64] FLAG: --version="false"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956274    2572 flags.go:64] FLAG: --vmodule=""
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956281    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.956286    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956463    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 07:38:12.964989 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956470    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956476    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956481    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956486    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956491    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956495    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956501    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956506    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956510    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956514    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956518    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956522    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956526    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956530    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956534    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956538    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956542    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956547    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956551    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956555    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 07:38:12.965586 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956560    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956564    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956568    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956573    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956577    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956581    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956585    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956589    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956593    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956598    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956602    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956607    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956611    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956615    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956619    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956623    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956630    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956636    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956642    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 07:38:12.966049 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956647    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956651    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956655    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956659    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956663    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956667    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956671    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956675    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956679    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956683    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956687    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956691    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956696    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956700    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956705    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956709    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956713    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956717    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956721    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956725    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 07:38:12.966537 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956731    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956735    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956740    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956745    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956750    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956754    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956758    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956762    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956766    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956770    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956774    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956780    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956784    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956789    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956793    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956797    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956803    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956808    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956813    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 07:38:12.966990 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956818    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956822    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956827    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956831    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956835    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956840    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956843    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956848    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956852    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956856    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956860    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956866    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956872    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956878    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956882    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956886    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956891    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956900    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956905    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 07:38:12.967451 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.956909    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.957636    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965009    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965024    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965044    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965049    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965123    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965129    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965133    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965135    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965138    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965141    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965146    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965150    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 07:38:12.967893 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965153    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965156    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965159    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965161    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965164    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965167    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965169    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965172    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965174    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965177    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965180    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965182    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965185    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965188    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965190    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965193    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965196    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965199    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965202    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965204    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 07:38:12.968255 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965207    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965210    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965213    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965215    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965218    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965222    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965225    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965228    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965231    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965234    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965237    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965239    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965242    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965245    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965248    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965252    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965255    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965258    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965261    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 07:38:12.968721 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965263    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965266    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965268    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965271    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965273    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965276    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965278    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965281    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965283    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965285    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965289    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965292    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965294    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965296    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965299    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965302    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965304    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965307    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965310    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965312    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 07:38:12.969186 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965315    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965317    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965320    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965322    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965324    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965327    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965329    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965332    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965334    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965337    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965339    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965341    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965344    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965347    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965349    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965351    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965354    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965356    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965359    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965361    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 07:38:12.969656 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965364    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965366    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965368    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965372    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965374    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965376    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965379    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965382    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965384    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965387    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965389    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965392    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965397    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965520    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965524    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 07:38:12.970231 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965528    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965530    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965533    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965536    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965539    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965542    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965544    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965547    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965549    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965552    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965554    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965557    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965560    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965563    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965566    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965569    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965572    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965574    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965577    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 07:38:12.970585 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965579    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965582    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965585    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965587    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965590    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965593    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965596    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965599    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965602    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965604    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965607    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965609    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965612    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965614    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965617    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965619    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965621    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965624    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965626    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 07:38:12.971031 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965629    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965632    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965634    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965637    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965639    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965641    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965645    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965648    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965651    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965654    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965656    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965659    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965661    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965664    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965666    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965669    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965671    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965674    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965677    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965680    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 07:38:12.971488 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965682    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965685    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965687    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965690    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965692    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965694    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965697    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965699    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965701    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965704    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965706    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965709    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965711    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965714    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965716    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965718    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965721    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965724    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965726    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965728    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 07:38:12.971953 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965731    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965733    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965736    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965738    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965740    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965743    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965746    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965749    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965752    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965756    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965761    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965764    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965766    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965769    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965772    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965774    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965777    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965779    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 07:38:12.972524 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:12.965782    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 07:38:12.972941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965786    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 11 07:38:12.972941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.965890    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 11 07:38:12.972941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.969112    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 11 07:38:12.972941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.970626    2572 server.go:1019] "Starting client certificate rotation"
Jul 11 07:38:12.972941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.970724    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 11 07:38:12.972941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.970751    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 11 07:38:12.973119 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.972960    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 11 07:38:12.974426 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.974409    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 11 07:38:12.986036 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.986000    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 11 07:38:12.988161 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.988144    2572 log.go:25] "Validated CRI v1 image API"
Jul 11 07:38:12.988898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.988883    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 11 07:38:12.991984 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.991963    2572 fs.go:135] Filesystem UUIDs: map[1c1448af-0c7f-40a5-b0c0-894781252098:/dev/nvme0n1p4 76dd5631-90b7-4974-bc6f-6fe160136187:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 11 07:38:12.992053 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.991982    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 11 07:38:12.996857 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.996737    2572 manager.go:216] Machine: {Timestamp:2026-07-11 07:38:12.995481977 +0000 UTC m=+0.310336876 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203932 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec281356bd47796259454bfcc04627b5 SystemUUID:ec281356-bd47-7962-5945-4bfcc04627b5 BootID:d5bd029d-0bb7-4fa3-874d-23cd22482f8d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:a8:b7:1d:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:a8:b7:1d:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:08:91:ea:9e:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 11 07:38:12.996857 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.996849    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 11 07:38:12.997021 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:12.997004    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 11 07:38:13.000134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.000106    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 11 07:38:13.000276 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.000135    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-181.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 11 07:38:13.000748 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.000738    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 11 07:38:13.000783 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.000751    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 11 07:38:13.000783 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.000758    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 11 07:38:13.001427 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.001417    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 11 07:38:13.002429 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.002413    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 11 07:38:13.003245 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.003235    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 11 07:38:13.006879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.006868    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 11 07:38:13.006993 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.006984    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 11 07:38:13.007027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007013    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 11 07:38:13.007027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007025    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 11 07:38:13.007082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007048    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 11 07:38:13.007082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007061    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 11 07:38:13.007082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007072    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 11 07:38:13.007480 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007468    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 11 07:38:13.007521 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.007488    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 11 07:38:13.009391 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.009373    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 11 07:38:13.011385 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011370    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 11 07:38:13.011468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011406    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 11 07:38:13.011591 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011578    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 11 07:38:13.011651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011598    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 11 07:38:13.011651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011607    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 11 07:38:13.011651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011615    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 11 07:38:13.011651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011629    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 11 07:38:13.011651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011637    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 11 07:38:13.011965 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011955    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 11 07:38:13.012015 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011967    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 11 07:38:13.012015 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011977    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 11 07:38:13.012015 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.011987    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 11 07:38:13.012015 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.012006    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 11 07:38:13.012159 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.012020    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 11 07:38:13.012667 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.012656    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 11 07:38:13.012716 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.012671    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 11 07:38:13.015362 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.015315    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-181.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 11 07:38:13.015748 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.015736    2572 server.go:1268] "Started kubelet"
Jul 11 07:38:13.017934 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:13.017526    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 11 07:38:13.017934 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:13.017575    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-181.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 11 07:38:13.017934 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.015820    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 11 07:38:13.017934 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.017783    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 11 07:38:13.018172 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.017949    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 11 07:38:13.018231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.018171    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zng7f"
Jul 11 07:38:13.018265 ip-10-0-141-181 systemd[1]: Started Kubernetes Kubelet.
Jul 11 07:38:13.020511 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.020491    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 11 07:38:13.021050 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.021038    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 11 07:38:13.022947 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:13.022909    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 11 07:38:13.024183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:13.024164    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zng7f"
Jul 11 07:38:13.025437 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:13.024536    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-181.ec2.internal.18c12c2b8f5a6035  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-181.ec2.internal,UID:ip-10-0-141-181.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-181.ec2.internal,},FirstTimestamp:2026-07-11 07:38:13.015707701 +0000 UTC m=+0.330562579,LastTimestamp:2026-07-11 07:38:13.015707701 +0000 UTC m=+0.330562579,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-181.ec2.internal,}"
Jul 11 07:38:14.139503 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.139491    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 11 07:38:14.139635 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.139493    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 11 07:38:14.139719 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.139699    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.140265 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140246    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 11 07:38:14.140341 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140266    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 11 07:38:14.140341 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140280    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 11 07:38:14.140438 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140366    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 11 07:38:14.140438 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140374    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 11 07:38:14.140438 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140400    2572 factory.go:55] Registering systemd factory
Jul 11 07:38:14.140438 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140413    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 11 07:38:14.140670 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140629    2572 factory.go:153] Registering CRI-O factory
Jul 11 07:38:14.140670 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140642    2572 factory.go:223] Registration of the crio container factory successfully
Jul 11 07:38:14.140752 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140685    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 11 07:38:14.140752 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140705    2572 factory.go:103] Registering Raw factory
Jul 11 07:38:14.140752 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.140720    2572 manager.go:1196] Started watching for new ooms in manager
Jul 11 07:38:14.141090 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.141076    2572 manager.go:349] Starting recovery of all containers
Jul 11 07:38:14.141312 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.141285    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 11 07:38:14.141312 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.141286    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-181.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 11 07:38:14.152481 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.152463    2572 manager.go:354] Recovery completed
Jul 11 07:38:14.156811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.156799    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.158651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.158637    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.158706 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.158663    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.158706 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.158673    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.159146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159132    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 11 07:38:14.159146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159145    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 11 07:38:14.159249 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159162    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 11 07:38:14.159249 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159138    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.159949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159931    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.160018 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159960    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.160018 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.159978    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.160610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.160593    2572 policy_none.go:49] "None policy: Start"
Jul 11 07:38:14.160610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.160612    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 11 07:38:14.160706 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.160623    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 11 07:38:14.161757 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.161746    2572 policy_none.go:47] "Start"
Jul 11 07:38:14.205091 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205077    2572 manager.go:337] "Starting Device Plugin manager"
Jul 11 07:38:14.205177 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.205108    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 11 07:38:14.205177 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205117    2572 server.go:87] "Starting device plugin registration server"
Jul 11 07:38:14.205372 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205361    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 11 07:38:14.205409 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205375    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 11 07:38:14.205477 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205454    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 11 07:38:14.205576 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205560    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 11 07:38:14.205576 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.205573    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 11 07:38:14.206006 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.205989    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 11 07:38:14.206072 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.206029    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.286889 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.286855    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 11 07:38:14.288052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.287952    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 11 07:38:14.288052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.287982    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 11 07:38:14.288052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.288007    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 11 07:38:14.288052 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.288048    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 11 07:38:14.290225 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.290208    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 07:38:14.306056 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.306009    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.306805 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.306792    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.306879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.306818    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.306879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.306829    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.306879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.306850    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.313769 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.313755    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.313808 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.313776    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-181.ec2.internal\": node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.326459 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.326438    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.388828 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.388798    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"]
Jul 11 07:38:14.388906 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.388872    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.389596 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.389583    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.389653 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.389608    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.389653 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.389618    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.390922 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.390911    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.391032 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.391019    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.391068 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.391046    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.391613 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.391599    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.391677 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.391624    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.391677 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.391634    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.392037 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.392019    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.392117 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.392048    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.392117 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.392058    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.393301 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.393288    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.393366 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.393311    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 07:38:14.393936 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.393921    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 07:38:14.393997 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.393947    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 07:38:14.393997 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.393960    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeHasSufficientPID"
Jul 11 07:38:14.417982 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.417966    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-181.ec2.internal\" not found" node="ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.422048 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.422036    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-181.ec2.internal\" not found" node="ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.427091 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.427077    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.442189 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.442168    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/913bbc9bdebf5b8c197c8e2f6171a3a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal\" (UID: \"913bbc9bdebf5b8c197c8e2f6171a3a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.442257 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.442191    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/913bbc9bdebf5b8c197c8e2f6171a3a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal\" (UID: \"913bbc9bdebf5b8c197c8e2f6171a3a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.442257 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.442209    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f661395125ffa7bd7cd96cb8f2630ea-config\") pod \"kube-apiserver-proxy-ip-10-0-141-181.ec2.internal\" (UID: \"6f661395125ffa7bd7cd96cb8f2630ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.528199 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.528171    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.542580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.542499    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/913bbc9bdebf5b8c197c8e2f6171a3a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal\" (UID: \"913bbc9bdebf5b8c197c8e2f6171a3a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.542657 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.542604    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/913bbc9bdebf5b8c197c8e2f6171a3a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal\" (UID: \"913bbc9bdebf5b8c197c8e2f6171a3a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.542657 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.542623    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f661395125ffa7bd7cd96cb8f2630ea-config\") pod \"kube-apiserver-proxy-ip-10-0-141-181.ec2.internal\" (UID: \"6f661395125ffa7bd7cd96cb8f2630ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.542657 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.542555    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/913bbc9bdebf5b8c197c8e2f6171a3a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal\" (UID: \"913bbc9bdebf5b8c197c8e2f6171a3a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.542657 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.542655    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f661395125ffa7bd7cd96cb8f2630ea-config\") pod \"kube-apiserver-proxy-ip-10-0-141-181.ec2.internal\" (UID: \"6f661395125ffa7bd7cd96cb8f2630ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.542790 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.542676    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/913bbc9bdebf5b8c197c8e2f6171a3a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal\" (UID: \"913bbc9bdebf5b8c197c8e2f6171a3a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.629066 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.628981    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.720613 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.720568    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.725945 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:14.725927    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:14.729078 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.729063    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.829625 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.829589    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:14.930097 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:14.930040    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.030614 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.030580    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.081054 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.081026    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 11 07:38:15.081502 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.081171    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 11 07:38:15.131231 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.131206    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.142060 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.142032    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-10 07:33:14 +0000 UTC" deadline="2028-01-19 15:26:08.216144261 +0000 UTC"
Jul 11 07:38:15.142060 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.142057    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13375h47m53.074089934s"
Jul 11 07:38:15.231943 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.231918    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.232618 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:15.232588    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f661395125ffa7bd7cd96cb8f2630ea.slice/crio-202727ef0dae559abf654731f93ac95f4ef776c88d87b2fc0a088d9896c50609 WatchSource:0}: Error finding container 202727ef0dae559abf654731f93ac95f4ef776c88d87b2fc0a088d9896c50609: Status 404 returned error can't find the container with id 202727ef0dae559abf654731f93ac95f4ef776c88d87b2fc0a088d9896c50609
Jul 11 07:38:15.232891 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:15.232874    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913bbc9bdebf5b8c197c8e2f6171a3a4.slice/crio-413e47773028be743314864ea65edd754d25eee0abe00973b793610bb7c423ca WatchSource:0}: Error finding container 413e47773028be743314864ea65edd754d25eee0abe00973b793610bb7c423ca: Status 404 returned error can't find the container with id 413e47773028be743314864ea65edd754d25eee0abe00973b793610bb7c423ca
Jul 11 07:38:15.237088 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.237076    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 07:38:15.291950 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.291897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal" event={"ID":"913bbc9bdebf5b8c197c8e2f6171a3a4","Type":"ContainerStarted","Data":"413e47773028be743314864ea65edd754d25eee0abe00973b793610bb7c423ca"}
Jul 11 07:38:15.292908 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.292878    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal" event={"ID":"6f661395125ffa7bd7cd96cb8f2630ea","Type":"ContainerStarted","Data":"202727ef0dae559abf654731f93ac95f4ef776c88d87b2fc0a088d9896c50609"}
Jul 11 07:38:15.332014 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.331975    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.432560 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.432508    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.533028 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.532960    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.539963 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.539946    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 07:38:15.575111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.575087    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 07:38:15.633058 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:15.633030    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-181.ec2.internal\" not found"
Jul 11 07:38:15.660851 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.660825    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 07:38:15.739740 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.739715    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:15.749550 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.749513    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 11 07:38:15.750461 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.750443    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal"
Jul 11 07:38:15.758125 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:15.758106    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 11 07:38:16.119623 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.119596    2572 apiserver.go:52] "Watching apiserver"
Jul 11 07:38:16.126706 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.126680    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 11 07:38:16.127734 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.127710    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-8lm75","kube-system/konnectivity-agent-crg4g","kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r","openshift-cluster-node-tuning-operator/tuned-xtxg6","openshift-dns/node-resolver-jt4gq","openshift-image-registry/node-ca-g9gcz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal","openshift-multus/multus-additional-cni-plugins-lh56c","kube-system/global-pull-secret-syncer-pmtjg","openshift-multus/multus-d4gvg","openshift-multus/network-metrics-daemon-jnj7r","openshift-network-diagnostics/network-check-target-ngmk7","openshift-network-operator/iptables-alerter-pnkqs"]
Jul 11 07:38:16.129086 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.129067    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.130191 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.130166    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:16.130284 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.130240    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:16.131200 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.131062    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pzd94\""
Jul 11 07:38:16.131328 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.131243    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.131389 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.131366    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 11 07:38:16.131571 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.131544    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.131654 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.131600    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.132785 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.132765    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.133552 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.133518    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 11 07:38:16.133645 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.133600    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.133698 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.133678    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pbs5g\""
Jul 11 07:38:16.133943 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.133929    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.134214 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.134198    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.134777 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.134759    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zt2cg\""
Jul 11 07:38:16.134869 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.134790    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 11 07:38:16.135160 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.135142    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.135245 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.135199    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.135601 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.135581    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.136167 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.136135    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7h2x7\""
Jul 11 07:38:16.136803 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.136342    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.136803 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.136551    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.137050 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.137034    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.137339 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.137320    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-962mn\""
Jul 11 07:38:16.137625 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.137606    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.137708 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.137672    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.138649 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.138633    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.139325 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139310    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.139419 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139363    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 11 07:38:16.139616 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139584    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 11 07:38:16.139696 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139671    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.139696 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139684    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 11 07:38:16.139887 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139869    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 11 07:38:16.139964 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.139906    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bmrdz\""
Jul 11 07:38:16.140058 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.140037    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.141091 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.141033    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 11 07:38:16.141091 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.141086    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 11 07:38:16.141836 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.141819    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r68tw\""
Jul 11 07:38:16.142058 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.142042    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 11 07:38:16.142146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.142086    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 11 07:38:16.142146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.142099    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 11 07:38:16.142146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.142130    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 11 07:38:16.143246 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.143227    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.143330 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.143309    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:16.144645 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.144109    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tm652\""
Jul 11 07:38:16.145885 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.144962    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 11 07:38:16.145885 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.145110    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 11 07:38:16.146380 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.146365    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.148399 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.148381    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.148499 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.148440    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:16.148734 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.148672    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 11 07:38:16.148734 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.148688    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g7r2d\""
Jul 11 07:38:16.151479 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151459    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-cni-binary-copy\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.151580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151492    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysctl-conf\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.151580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151540    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9lhh\" (UniqueName: \"kubernetes.io/projected/3ebd9210-2432-4736-b8c2-84c298aedb32-kube-api-access-w9lhh\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.151580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151567    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-systemd-units\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.151733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151585    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-systemd\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.151733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.151733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151618    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-socket-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.151733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151676    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.151733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151708    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-registration-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151730    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-device-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151772    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42lbb\" (UniqueName: \"kubernetes.io/projected/fde0f7b1-e969-480a-8209-31641ab1966e-kube-api-access-42lbb\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151795    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-var-lib-kubelet\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151817    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/547f78c4-01e1-4826-8413-0685d61c4f36-tmp-dir\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151832    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-dbus\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151847    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-cnibin\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151867    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-var-lib-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151890    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151924    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64gq\" (UniqueName: \"kubernetes.io/projected/93354949-75ca-4479-a04b-1f74bdb0dd7a-kube-api-access-k64gq\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.151960 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151951    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovn-node-metrics-cert\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151980    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-etc-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.151985    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152014    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-kubelet-config\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152035    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5f4\" (UniqueName: \"kubernetes.io/projected/73b9ad31-5d49-4664-8636-c15d1a9794b1-kube-api-access-8z5f4\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152057    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152079    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9375fbe2-bd17-4abd-9ed0-9c9584f60ecd-agent-certs\") pod \"konnectivity-agent-crg4g\" (UID: \"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd\") " pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152099    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fde0f7b1-e969-480a-8209-31641ab1966e-host\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152120    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-run-netns\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152142    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-cni-netd\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152166    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152187    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fde0f7b1-e969-480a-8209-31641ab1966e-serviceca\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152224    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-node-log\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152287    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152312    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-lib-modules\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-cni-bin\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152362    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.152422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152393    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovnkube-script-lib\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152450    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64ngv\" (UniqueName: \"kubernetes.io/projected/a60fee7f-75a5-4e89-93f0-e8f5e571475a-kube-api-access-64ngv\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152489    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73b9ad31-5d49-4664-8636-c15d1a9794b1-iptables-alerter-script\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152563    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152610    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-log-socket\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-system-cni-dir\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152688    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysconfig\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152707    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxstq\" (UniqueName: \"kubernetes.io/projected/547f78c4-01e1-4826-8413-0685d61c4f36-kube-api-access-bxstq\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152724    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhvpr\" (UniqueName: \"kubernetes.io/projected/453778fe-8de5-49f4-b6dd-54592e220864-kube-api-access-rhvpr\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152764    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-os-release\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152802    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-kubernetes\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152820    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-systemd\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152841    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-run\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152861    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-tuned\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152898    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/547f78c4-01e1-4826-8413-0685d61c4f36-hosts-file\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152927    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-kubelet\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.153190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152947    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152961    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73b9ad31-5d49-4664-8636-c15d1a9794b1-host-slash\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.152977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153014    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysctl-d\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153035    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-sys\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153049    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-slash\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153071    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-modprobe-d\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153099    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-sys-fs\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9375fbe2-bd17-4abd-9ed0-9c9584f60ecd-konnectivity-ca\") pod \"konnectivity-agent-crg4g\" (UID: \"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd\") " pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153147    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-ovn\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovnkube-config\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153204    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-env-overrides\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153222    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-host\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.154027 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.153241    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebd9210-2432-4736-b8c2-84c298aedb32-tmp\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.168172 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.168156    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kmnnz"
Jul 11 07:38:16.175351 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.175319    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kmnnz"
Jul 11 07:38:16.241191 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.241166    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 11 07:38:16.253857 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253834    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-dbus\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253865    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-cni-binary-copy\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253898    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-hostroot\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253914    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-cnibin\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-var-lib-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253956    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.253979    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-conf-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.254014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-multus-certs\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254017    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-var-lib-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254026    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64gq\" (UniqueName: \"kubernetes.io/projected/93354949-75ca-4479-a04b-1f74bdb0dd7a-kube-api-access-k64gq\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254087    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovn-node-metrics-cert\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254117    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-etc-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254143    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-kubelet-config\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-dbus\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254170    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-cni-bin\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254197    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5f4\" (UniqueName: \"kubernetes.io/projected/73b9ad31-5d49-4664-8636-c15d1a9794b1-kube-api-access-8z5f4\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254200    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-etc-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254234    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254253    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-kubelet-config\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254260    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9375fbe2-bd17-4abd-9ed0-9c9584f60ecd-agent-certs\") pod \"konnectivity-agent-crg4g\" (UID: \"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd\") " pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254285    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fde0f7b1-e969-480a-8209-31641ab1966e-host\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254293    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254308    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-run-netns\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.254359 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254331    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-cni-netd\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254362    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-daemon-config\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-run-netns\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254432    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-cni-netd\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254458    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254475    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fde0f7b1-e969-480a-8209-31641ab1966e-host\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fde0f7b1-e969-480a-8209-31641ab1966e-serviceca\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254570    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-node-log\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254596    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254631    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-lib-modules\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-cni-bin\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254694    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254693    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254725    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovnkube-script-lib\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254755    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ngv\" (UniqueName: \"kubernetes.io/projected/a60fee7f-75a5-4e89-93f0-e8f5e571475a-kube-api-access-64ngv\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254786    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-cni-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-socket-dir-parent\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.255111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254846    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pfm4\" (UniqueName: \"kubernetes.io/projected/29f420da-3869-4a78-97a2-c4e78477cfb2-kube-api-access-8pfm4\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73b9ad31-5d49-4664-8636-c15d1a9794b1-iptables-alerter-script\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254911    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254938    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-log-socket\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254968    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-system-cni-dir\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255006    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysconfig\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255035    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxstq\" (UniqueName: \"kubernetes.io/projected/547f78c4-01e1-4826-8413-0685d61c4f36-kube-api-access-bxstq\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255065    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-k8s-cni-cncf-io\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255092    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-kubelet\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255115    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhvpr\" (UniqueName: \"kubernetes.io/projected/453778fe-8de5-49f4-b6dd-54592e220864-kube-api-access-rhvpr\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.254017    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-cnibin\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255149    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-lib-modules\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255173    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-system-cni-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255167    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255197    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-node-log\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255214    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-os-release\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255248    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-kubernetes\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.255894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255276    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-systemd\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255301    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-run\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.255278    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255332    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-os-release\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255339    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-tuned\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255388    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255395    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-systemd\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255429    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysconfig\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255434    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-cni-bin\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.255476    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:16.75543757 +0000 UTC m=+2.958856432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255478    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-run\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255518    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-kubernetes\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255577    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-log-socket\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255602    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fde0f7b1-e969-480a-8209-31641ab1966e-serviceca\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255621    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-system-cni-dir\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/547f78c4-01e1-4826-8413-0685d61c4f36-hosts-file\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255710    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.256603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255825    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-kubelet\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255855    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255870    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/547f78c4-01e1-4826-8413-0685d61c4f36-hosts-file\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255893    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255921    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-kubelet\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255964    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73b9ad31-5d49-4664-8636-c15d1a9794b1-host-slash\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255978    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovnkube-script-lib\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.255991    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256025    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73b9ad31-5d49-4664-8636-c15d1a9794b1-host-slash\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256027    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysctl-d\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256084    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-sys\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256122    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73b9ad31-5d49-4664-8636-c15d1a9794b1-iptables-alerter-script\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256146    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysctl-d\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256175    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-slash\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256215    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-sys\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256218    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-cnibin\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256255    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-os-release\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.257292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256260    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-host-slash\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256282    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-modprobe-d\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-sys-fs\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256340    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9375fbe2-bd17-4abd-9ed0-9c9584f60ecd-konnectivity-ca\") pod \"konnectivity-agent-crg4g\" (UID: \"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd\") " pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256399    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-modprobe-d\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256400    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-sys-fs\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256424    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-ovn\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovnkube-config\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256485    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-env-overrides\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256517    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np9d4\" (UniqueName: \"kubernetes.io/projected/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-kube-api-access-np9d4\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256568    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-host\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256592    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebd9210-2432-4736-b8c2-84c298aedb32-tmp\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-cni-binary-copy\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysctl-conf\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9lhh\" (UniqueName: \"kubernetes.io/projected/3ebd9210-2432-4736-b8c2-84c298aedb32-kube-api-access-w9lhh\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256683    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-systemd-units\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256732    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-systemd\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256780    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-systemd\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257134    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257178    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-socket-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257211    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257244    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-netns\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257271    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257292    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-cni-multus\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.256461    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-ovn\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-etc-kubernetes\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-registration-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257381    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-env-overrides\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257409    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-device-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257439    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42lbb\" (UniqueName: \"kubernetes.io/projected/fde0f7b1-e969-480a-8209-31641ab1966e-kube-api-access-42lbb\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovnkube-config\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257463    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-var-lib-kubelet\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257495    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/547f78c4-01e1-4826-8413-0685d61c4f36-tmp-dir\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257519    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-var-lib-kubelet\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.258845 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257619    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-run-openvswitch\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.257901    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/547f78c4-01e1-4826-8413-0685d61c4f36-tmp-dir\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258002    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-host\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258059    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9375fbe2-bd17-4abd-9ed0-9c9584f60ecd-konnectivity-ca\") pod \"konnectivity-agent-crg4g\" (UID: \"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd\") " pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258112    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-sysctl-conf\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258657    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/453778fe-8de5-49f4-b6dd-54592e220864-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258680    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-registration-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258686    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-device-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258759    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a60fee7f-75a5-4e89-93f0-e8f5e571475a-systemd-units\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258806    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/453778fe-8de5-49f4-b6dd-54592e220864-cni-binary-copy\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.259608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.259123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a60fee7f-75a5-4e89-93f0-e8f5e571475a-ovn-node-metrics-cert\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.260913 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.260544    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9375fbe2-bd17-4abd-9ed0-9c9584f60ecd-agent-certs\") pod \"konnectivity-agent-crg4g\" (UID: \"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd\") " pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.262138 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.258188    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93354949-75ca-4479-a04b-1f74bdb0dd7a-socket-dir\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.263247 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.263221    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ebd9210-2432-4736-b8c2-84c298aedb32-etc-tuned\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.263510 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.263488    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64gq\" (UniqueName: \"kubernetes.io/projected/93354949-75ca-4479-a04b-1f74bdb0dd7a-kube-api-access-k64gq\") pod \"aws-ebs-csi-driver-node-plk5r\" (UID: \"93354949-75ca-4479-a04b-1f74bdb0dd7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.264205 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.264184    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebd9210-2432-4736-b8c2-84c298aedb32-tmp\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.264326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.264307    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5f4\" (UniqueName: \"kubernetes.io/projected/73b9ad31-5d49-4664-8636-c15d1a9794b1-kube-api-access-8z5f4\") pod \"iptables-alerter-pnkqs\" (UID: \"73b9ad31-5d49-4664-8636-c15d1a9794b1\") " pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.266624 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.266604    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:16.266729 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.266632    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:16.266729 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.266646    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:16.266729 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.266712    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:16.766694463 +0000 UTC m=+2.970113320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:16.267635 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.267591    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhvpr\" (UniqueName: \"kubernetes.io/projected/453778fe-8de5-49f4-b6dd-54592e220864-kube-api-access-rhvpr\") pod \"multus-additional-cni-plugins-lh56c\" (UID: \"453778fe-8de5-49f4-b6dd-54592e220864\") " pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.268130 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.268111    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42lbb\" (UniqueName: \"kubernetes.io/projected/fde0f7b1-e969-480a-8209-31641ab1966e-kube-api-access-42lbb\") pod \"node-ca-g9gcz\" (UID: \"fde0f7b1-e969-480a-8209-31641ab1966e\") " pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.268770 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.268747    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxstq\" (UniqueName: \"kubernetes.io/projected/547f78c4-01e1-4826-8413-0685d61c4f36-kube-api-access-bxstq\") pod \"node-resolver-jt4gq\" (UID: \"547f78c4-01e1-4826-8413-0685d61c4f36\") " pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.268879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.268863    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9lhh\" (UniqueName: \"kubernetes.io/projected/3ebd9210-2432-4736-b8c2-84c298aedb32-kube-api-access-w9lhh\") pod \"tuned-xtxg6\" (UID: \"3ebd9210-2432-4736-b8c2-84c298aedb32\") " pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.269271 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.269241    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64ngv\" (UniqueName: \"kubernetes.io/projected/a60fee7f-75a5-4e89-93f0-e8f5e571475a-kube-api-access-64ngv\") pod \"ovnkube-node-8lm75\" (UID: \"a60fee7f-75a5-4e89-93f0-e8f5e571475a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.358816 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358789    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-daemon-config\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358833    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-cni-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358862    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-socket-dir-parent\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pfm4\" (UniqueName: \"kubernetes.io/projected/29f420da-3869-4a78-97a2-c4e78477cfb2-kube-api-access-8pfm4\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358915    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-k8s-cni-cncf-io\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358939    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-kubelet\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358943    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-cni-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.358986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358963    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-system-cni-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358993    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359001    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-kubelet\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359010    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-k8s-cni-cncf-io\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.358963    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-socket-dir-parent\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359043    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-cnibin\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359064    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-os-release\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359092    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np9d4\" (UniqueName: \"kubernetes.io/projected/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-kube-api-access-np9d4\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.359106    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359155    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-cnibin\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.359166    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:16.859150394 +0000 UTC m=+3.062569240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359193    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-netns\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359221    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-cni-multus\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-os-release\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359235    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-system-cni-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359255    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-netns\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359299    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-cni-multus\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.359335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359341    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-etc-kubernetes\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359372    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-cni-binary-copy\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359377    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-etc-kubernetes\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359401    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-hostroot\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359429    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-conf-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359437    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-daemon-config\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359484    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-multus-certs\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359489    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-hostroot\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359513    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-multus-conf-dir\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359513    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-cni-bin\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359581    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-run-multus-certs\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359619    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-host-var-lib-cni-bin\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.360097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.359866    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-cni-binary-copy\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.366632 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.366608    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np9d4\" (UniqueName: \"kubernetes.io/projected/9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd-kube-api-access-np9d4\") pod \"multus-d4gvg\" (UID: \"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd\") " pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.366729 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.366659    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pfm4\" (UniqueName: \"kubernetes.io/projected/29f420da-3869-4a78-97a2-c4e78477cfb2-kube-api-access-8pfm4\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.405404 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.405334    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 07:38:16.440033 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.440000    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g9gcz"
Jul 11 07:38:16.449107 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.449084    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pnkqs"
Jul 11 07:38:16.457799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.457780    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r"
Jul 11 07:38:16.462640 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.462622    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xtxg6"
Jul 11 07:38:16.469278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.469260    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jt4gq"
Jul 11 07:38:16.475863 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.475845    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lh56c"
Jul 11 07:38:16.482580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.482561    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:16.488267 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.488248    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:16.493304 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.493289    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4gvg"
Jul 11 07:38:16.763176 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.763104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:16.763319 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.763220    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:16.763319 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.763276    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:17.763260432 +0000 UTC m=+3.966679273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:16.863779 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.863726    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:16.863779 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:16.863780    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:16.864030 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.863890    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:16.864030 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.863937    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:16.864030 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.863951    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:16.864030 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.863955    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:17.86394026 +0000 UTC m=+4.067359115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:16.864030 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.863960    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:16.864030 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:16.864000    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:17.863985774 +0000 UTC m=+4.067404639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:17.026216 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.026188    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde0f7b1_e969_480a_8209_31641ab1966e.slice/crio-57d6ec79cce9a2312e57b3aaaae9b626417515c7845dde0c31e7e543a7424a96 WatchSource:0}: Error finding container 57d6ec79cce9a2312e57b3aaaae9b626417515c7845dde0c31e7e543a7424a96: Status 404 returned error can't find the container with id 57d6ec79cce9a2312e57b3aaaae9b626417515c7845dde0c31e7e543a7424a96
Jul 11 07:38:17.027954 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.027927    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4eea64_5cb7_45d0_8d2f_2e5f48d091bd.slice/crio-fedb6e5ddfbee150b6ce0e27ad1c0d4173c6a1cec2b9332969a412ae9a6f62c6 WatchSource:0}: Error finding container fedb6e5ddfbee150b6ce0e27ad1c0d4173c6a1cec2b9332969a412ae9a6f62c6: Status 404 returned error can't find the container with id fedb6e5ddfbee150b6ce0e27ad1c0d4173c6a1cec2b9332969a412ae9a6f62c6
Jul 11 07:38:17.031338 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.031318    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9375fbe2_bd17_4abd_9ed0_9c9584f60ecd.slice/crio-e0df61883ebfe953dea61fe43e79892b4d1cd489a6fd3728cfbd815d5b5bcf25 WatchSource:0}: Error finding container e0df61883ebfe953dea61fe43e79892b4d1cd489a6fd3728cfbd815d5b5bcf25: Status 404 returned error can't find the container with id e0df61883ebfe953dea61fe43e79892b4d1cd489a6fd3728cfbd815d5b5bcf25
Jul 11 07:38:17.032033 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.032014    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebd9210_2432_4736_b8c2_84c298aedb32.slice/crio-331b82bb8a37027cdcdd2ee13619e995ac9d874d601928a378602fc2be41f5a0 WatchSource:0}: Error finding container 331b82bb8a37027cdcdd2ee13619e995ac9d874d601928a378602fc2be41f5a0: Status 404 returned error can't find the container with id 331b82bb8a37027cdcdd2ee13619e995ac9d874d601928a378602fc2be41f5a0
Jul 11 07:38:17.032860 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.032791    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93354949_75ca_4479_a04b_1f74bdb0dd7a.slice/crio-2b18bb068808482bf90bde2d5cc8b057d1f365fcc3008e6ebb616a4f18bf5fd4 WatchSource:0}: Error finding container 2b18bb068808482bf90bde2d5cc8b057d1f365fcc3008e6ebb616a4f18bf5fd4: Status 404 returned error can't find the container with id 2b18bb068808482bf90bde2d5cc8b057d1f365fcc3008e6ebb616a4f18bf5fd4
Jul 11 07:38:17.033608 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.033587    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547f78c4_01e1_4826_8413_0685d61c4f36.slice/crio-f0d5796b30e8f0e309c9e3590afe4e8949f941036d078f85391e51905bbcad34 WatchSource:0}: Error finding container f0d5796b30e8f0e309c9e3590afe4e8949f941036d078f85391e51905bbcad34: Status 404 returned error can't find the container with id f0d5796b30e8f0e309c9e3590afe4e8949f941036d078f85391e51905bbcad34
Jul 11 07:38:17.034725 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.034702    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b9ad31_5d49_4664_8636_c15d1a9794b1.slice/crio-cc13a4e525b8bd1dbd7c42bae23a91aa8cf1db49db9cd111fe11ab70f90b7fa8 WatchSource:0}: Error finding container cc13a4e525b8bd1dbd7c42bae23a91aa8cf1db49db9cd111fe11ab70f90b7fa8: Status 404 returned error can't find the container with id cc13a4e525b8bd1dbd7c42bae23a91aa8cf1db49db9cd111fe11ab70f90b7fa8
Jul 11 07:38:17.035348 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.035318    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453778fe_8de5_49f4_b6dd_54592e220864.slice/crio-563a6491523c799f04148eee4dd1def09af3e8db7433d688b02b13524c8cc3e8 WatchSource:0}: Error finding container 563a6491523c799f04148eee4dd1def09af3e8db7433d688b02b13524c8cc3e8: Status 404 returned error can't find the container with id 563a6491523c799f04148eee4dd1def09af3e8db7433d688b02b13524c8cc3e8
Jul 11 07:38:17.037028 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:17.037013    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60fee7f_75a5_4e89_93f0_e8f5e571475a.slice/crio-78ad9ddac3ae2190916133c6e4c962d4f5b1545fca452e7f629723fe18f34ef7 WatchSource:0}: Error finding container 78ad9ddac3ae2190916133c6e4c962d4f5b1545fca452e7f629723fe18f34ef7: Status 404 returned error can't find the container with id 78ad9ddac3ae2190916133c6e4c962d4f5b1545fca452e7f629723fe18f34ef7
Jul 11 07:38:17.178835 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.178659    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 07:33:16 +0000 UTC" deadline="2028-02-01 09:50:10.559824647 +0000 UTC"
Jul 11 07:38:17.178835 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.178830    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13682h11m53.380998025s"
Jul 11 07:38:17.289065 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.288988    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:17.289187 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.289094    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:17.289187 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.289151    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:17.289268 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.289238    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:17.296504 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.296480    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"78ad9ddac3ae2190916133c6e4c962d4f5b1545fca452e7f629723fe18f34ef7"}
Jul 11 07:38:17.297676 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.297648    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerStarted","Data":"563a6491523c799f04148eee4dd1def09af3e8db7433d688b02b13524c8cc3e8"}
Jul 11 07:38:17.298674 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.298652    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crg4g" event={"ID":"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd","Type":"ContainerStarted","Data":"e0df61883ebfe953dea61fe43e79892b4d1cd489a6fd3728cfbd815d5b5bcf25"}
Jul 11 07:38:17.299606 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.299588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4gvg" event={"ID":"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd","Type":"ContainerStarted","Data":"fedb6e5ddfbee150b6ce0e27ad1c0d4173c6a1cec2b9332969a412ae9a6f62c6"}
Jul 11 07:38:17.301229 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.301208    2572 generic.go:356] "Generic (PLEG): container finished" podID="913bbc9bdebf5b8c197c8e2f6171a3a4" containerID="536e4890e0446e40fddccd46414b356d8918285b227db04ee0949dbd5f90245a" exitCode=0
Jul 11 07:38:17.301352 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.301268    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal" event={"ID":"913bbc9bdebf5b8c197c8e2f6171a3a4","Type":"ContainerDied","Data":"536e4890e0446e40fddccd46414b356d8918285b227db04ee0949dbd5f90245a"}
Jul 11 07:38:17.302577 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.302557    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal" event={"ID":"6f661395125ffa7bd7cd96cb8f2630ea","Type":"ContainerStarted","Data":"eaafcee1a978a8a33ed962d45378fe7dedf0771a7da6b632eae54eec9ffc9f40"}
Jul 11 07:38:17.303496 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.303473    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pnkqs" event={"ID":"73b9ad31-5d49-4664-8636-c15d1a9794b1","Type":"ContainerStarted","Data":"cc13a4e525b8bd1dbd7c42bae23a91aa8cf1db49db9cd111fe11ab70f90b7fa8"}
Jul 11 07:38:17.304520 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.304501    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jt4gq" event={"ID":"547f78c4-01e1-4826-8413-0685d61c4f36","Type":"ContainerStarted","Data":"f0d5796b30e8f0e309c9e3590afe4e8949f941036d078f85391e51905bbcad34"}
Jul 11 07:38:17.305415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.305398    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r" event={"ID":"93354949-75ca-4479-a04b-1f74bdb0dd7a","Type":"ContainerStarted","Data":"2b18bb068808482bf90bde2d5cc8b057d1f365fcc3008e6ebb616a4f18bf5fd4"}
Jul 11 07:38:17.306406 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.306388    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xtxg6" event={"ID":"3ebd9210-2432-4736-b8c2-84c298aedb32","Type":"ContainerStarted","Data":"331b82bb8a37027cdcdd2ee13619e995ac9d874d601928a378602fc2be41f5a0"}
Jul 11 07:38:17.307310 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.307290    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g9gcz" event={"ID":"fde0f7b1-e969-480a-8209-31641ab1966e","Type":"ContainerStarted","Data":"57d6ec79cce9a2312e57b3aaaae9b626417515c7845dde0c31e7e543a7424a96"}
Jul 11 07:38:17.327185 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.326690    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-181.ec2.internal" podStartSLOduration=2.326673243 podStartE2EDuration="2.326673243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:17.326599519 +0000 UTC m=+3.530018384" watchObservedRunningTime="2026-07-11 07:38:17.326673243 +0000 UTC m=+3.530092106"
Jul 11 07:38:17.771994 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.771907    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:17.772150 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.772107    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:17.772204 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.772169    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:19.772151161 +0000 UTC m=+5.975570008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:17.872832 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.872799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:17.873002 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:17.872842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:17.873081 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.873002    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:17.873081 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.873022    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:17.873081 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.873035    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:17.873230 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.873092    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:19.873073272 +0000 UTC m=+6.076492116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:17.873475 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.873459    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:17.873581 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:17.873508    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:19.87349363 +0000 UTC m=+6.076912475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:18.179779 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:18.179668    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 07:33:16 +0000 UTC" deadline="2028-03-21 00:43:26.521170478 +0000 UTC"
Jul 11 07:38:18.179779 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:18.179731    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14849h5m8.34144349s"
Jul 11 07:38:18.290387 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:18.290342    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:18.290579 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:18.290489    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:18.329485 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:18.329448    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal" event={"ID":"913bbc9bdebf5b8c197c8e2f6171a3a4","Type":"ContainerStarted","Data":"4526c1111c142d87df98755b88ee037d0551dee649f2fb7b9ea1883f96ed5f5a"}
Jul 11 07:38:19.288315 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:19.288280    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:19.288922 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.288444    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:19.288922 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:19.288828    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:19.289048 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.288920    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:19.790189 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:19.789625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:19.790189 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.789801    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:19.790189 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.789856    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:23.78983928 +0000 UTC m=+9.993258142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:19.890219 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:19.890184    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:19.890388 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:19.890230    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:19.890388 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.890376    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:19.890495 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.890395    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:19.890495 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.890408    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:19.890495 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.890413    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:19.890495 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.890461    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:23.89044264 +0000 UTC m=+10.093861484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:19.890495 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:19.890480    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:23.890470119 +0000 UTC m=+10.093888964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:20.289091 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:20.288615    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:20.289091 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:20.288732    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:21.288666 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:21.288631    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:21.288851 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:21.288761    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:21.289176 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:21.289027    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:21.289176 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:21.289135    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:22.288981 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:22.288948    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:22.289150 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:22.289084    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:23.289061 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:23.289029    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:23.289513 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.289176    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:23.289513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:23.289276    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:23.289513 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.289401    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:23.824649 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:23.824505    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:23.824809 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.824701    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:23.824809 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.824766    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:31.824749438 +0000 UTC m=+18.028168288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:23.925231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:23.925101    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:23.925231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:23.925145    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:23.925473 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.925276    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:23.925473 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.925279    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:23.925473 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.925357    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:31.925334017 +0000 UTC m=+18.128752870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:23.925473 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.925293    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:23.925473 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.925385    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:23.925473 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:23.925435    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:31.925419779 +0000 UTC m=+18.128838633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:24.290120 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:24.290041    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:24.290565 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:24.290157    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:25.288454 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:25.288418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:25.288454 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:25.288440    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:25.288703 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:25.288593    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:25.288770 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:25.288749    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:26.289044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:26.288836    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:26.289044 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:26.288971    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:27.288384 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:27.288349    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:27.288594 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:27.288349    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:27.288594 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:27.288482    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:27.288594 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:27.288548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:28.289115 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:28.289075    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:28.289582 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:28.289223    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:29.289009 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:29.288972    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:29.289190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:29.288973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:29.289190 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:29.289107    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:29.289190 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:29.289159    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:30.289130 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:30.289092    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:30.289311 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:30.289224    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:31.288801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:31.288728    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:31.288801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:31.288767    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:31.289069 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.288857    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:31.289069 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.289000    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:31.882366 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:31.882331    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:31.882892 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.882507    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:31.882892 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.882593    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:47.882569571 +0000 UTC m=+34.085988425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:31.983161 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:31.983130    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:31.983161 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:31.983161    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:31.983392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.983260    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:31.983392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.983273    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:31.983392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.983289    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:31.983392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.983300    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:31.983392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.983323    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:47.983305508 +0000 UTC m=+34.186724371 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:31.983392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:31.983340    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:47.983330966 +0000 UTC m=+34.186749813 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:32.288764 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:32.288670    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:32.288931 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:32.288794    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:33.288860 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:33.288820    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:33.289314 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:33.289245    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:33.289380 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:33.289332    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:33.289484 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:33.289464    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:34.289791 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:34.289756    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:34.290248 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:34.289864    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:35.289066 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:35.289032    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:35.289222 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:35.289032    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:35.289222 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:35.289171    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:35.289222 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:35.289196    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:36.289006 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.288644    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:36.289783 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:36.289115    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:36.372852 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.372818    2572 generic.go:356] "Generic (PLEG): container finished" podID="453778fe-8de5-49f4-b6dd-54592e220864" containerID="4c25f19cb374e6a5caca2b236f40025c6038c3d0d8bab33bef95ff6cd58b1e2c" exitCode=0
Jul 11 07:38:36.373028 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.372915    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerDied","Data":"4c25f19cb374e6a5caca2b236f40025c6038c3d0d8bab33bef95ff6cd58b1e2c"}
Jul 11 07:38:36.374898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.374875    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crg4g" event={"ID":"9375fbe2-bd17-4abd-9ed0-9c9584f60ecd","Type":"ContainerStarted","Data":"c9d78fe5c020d166e555f892c633be715cb9782c85506e5ab395fff0625bed24"}
Jul 11 07:38:36.376953 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.376934    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4gvg" event={"ID":"9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd","Type":"ContainerStarted","Data":"afa50d7eebc5a1928311bb7c5495237fa54742d589e5233ac88d04756840f715"}
Jul 11 07:38:36.378541 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.378502    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jt4gq" event={"ID":"547f78c4-01e1-4826-8413-0685d61c4f36","Type":"ContainerStarted","Data":"b6181da777a805db077935aa2d4876c3370d6c4df6bf0e84424307a3070ff4a8"}
Jul 11 07:38:36.380580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.380557    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r" event={"ID":"93354949-75ca-4479-a04b-1f74bdb0dd7a","Type":"ContainerStarted","Data":"ace0a2807a19b5e579465c2b50d316e15c99fd7a733286ba211e53d1fc349f2f"}
Jul 11 07:38:36.382171 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.382147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xtxg6" event={"ID":"3ebd9210-2432-4736-b8c2-84c298aedb32","Type":"ContainerStarted","Data":"c2d7a00b076d73ccfcf4d2839f0ab5388314964e4e1a2e7ef1fca69595aec8eb"}
Jul 11 07:38:36.383950 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.383829    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g9gcz" event={"ID":"fde0f7b1-e969-480a-8209-31641ab1966e","Type":"ContainerStarted","Data":"82827aa4dd83a0f2b7e176effe38269bad23f7c2430762902b889a42f70fb4a2"}
Jul 11 07:38:36.386934 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.386918    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:38:36.387278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387256    2572 generic.go:356] "Generic (PLEG): container finished" podID="a60fee7f-75a5-4e89-93f0-e8f5e571475a" containerID="5a648a288701a9d906401c5d5774430daf26eeec532f90843532f0d1dafefc7f" exitCode=1
Jul 11 07:38:36.387362 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387291    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"b88e18e1a794cdb9e57eba4b7e295b34cdaed9e8e91d9e1d608f9a02a7e079aa"}
Jul 11 07:38:36.387362 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387357    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"fb9d2848430954033e1bacb33142cdeb90ecc16cbca4be9b553c64510938292d"}
Jul 11 07:38:36.387480 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387406    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"3fc3719bbf015e4d1e5386de3ef79dbfd407615bb48da05d7fcf985a72d93093"}
Jul 11 07:38:36.387480 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387454    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"c50c49562d232c9d237e7ea56b35248e9a2632c28d63d173b1057cbf27829753"}
Jul 11 07:38:36.387604 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387500    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerDied","Data":"5a648a288701a9d906401c5d5774430daf26eeec532f90843532f0d1dafefc7f"}
Jul 11 07:38:36.387659 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.387606    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"11d14b8fee6a71cb659ce9c1ba6a7adb98fd039478e4c0b43aab7620f1a58953"}
Jul 11 07:38:36.394474 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.394433    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-181.ec2.internal" podStartSLOduration=21.39442124 podStartE2EDuration="21.39442124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:18.342465618 +0000 UTC m=+4.545884483" watchObservedRunningTime="2026-07-11 07:38:36.39442124 +0000 UTC m=+22.597840102"
Jul 11 07:38:36.421043 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.420991    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g9gcz" podStartSLOduration=4.090910554 podStartE2EDuration="22.420977286s" totalImagesPullingTime="18.330066732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:36.407098245 +0000 UTC m=+22.610517109" watchObservedRunningTime="2026-07-11 07:38:36.420977286 +0000 UTC m=+22.624396147"
Jul 11 07:38:36.432899 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.432824    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xtxg6" podStartSLOduration=4.088311278 podStartE2EDuration="22.432810575s" totalImagesPullingTime="18.344499297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:36.420926556 +0000 UTC m=+22.624345418" watchObservedRunningTime="2026-07-11 07:38:36.432810575 +0000 UTC m=+22.636229437"
Jul 11 07:38:36.433006 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.432907    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jt4gq" podStartSLOduration=4.022419794 podStartE2EDuration="22.432903259s" totalImagesPullingTime="18.410483465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:36.432452208 +0000 UTC m=+22.635871071" watchObservedRunningTime="2026-07-11 07:38:36.432903259 +0000 UTC m=+22.636322121"
Jul 11 07:38:36.444397 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.444359    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-crg4g" podStartSLOduration=4.098869742 podStartE2EDuration="22.444347492s" totalImagesPullingTime="18.34547775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:36.444131347 +0000 UTC m=+22.647550209" watchObservedRunningTime="2026-07-11 07:38:36.444347492 +0000 UTC m=+22.647766354"
Jul 11 07:38:36.458550 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.458497    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d4gvg" podStartSLOduration=4.046103079 podStartE2EDuration="22.458483759s" totalImagesPullingTime="18.41238068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:36.457990301 +0000 UTC m=+22.661409160" watchObservedRunningTime="2026-07-11 07:38:36.458483759 +0000 UTC m=+22.661902624"
Jul 11 07:38:36.653335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:36.653299    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 11 07:38:37.220782 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.220661    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-11T07:38:36.653318349Z","UUID":"ed054aea-5ee2-41f8-8140-ba3f4458ea5a","Handler":null,"Name":"","Endpoint":""}
Jul 11 07:38:37.222407 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.222388    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 11 07:38:37.222559 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.222427    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 11 07:38:37.288768 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.288737    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:37.289068 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.288739    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:37.289068 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:37.288845    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:37.289068 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:37.288966    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:37.391344 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.391310    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pnkqs" event={"ID":"73b9ad31-5d49-4664-8636-c15d1a9794b1","Type":"ContainerStarted","Data":"00763148cd52ccebf197091364965d827e4d3e1ed39f39bad2cc55c732d5ca52"}
Jul 11 07:38:37.393639 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.393588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r" event={"ID":"93354949-75ca-4479-a04b-1f74bdb0dd7a","Type":"ContainerStarted","Data":"0566b62bfcec927f2b10589d45756b91155149796c3e8633ec08fa6491eb2177"}
Jul 11 07:38:37.405145 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:37.405091    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pnkqs" podStartSLOduration=5.00038178 podStartE2EDuration="23.405074869s" totalImagesPullingTime="18.404693089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:37.404333998 +0000 UTC m=+23.607752861" watchObservedRunningTime="2026-07-11 07:38:37.405074869 +0000 UTC m=+23.608493726"
Jul 11 07:38:38.288509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:38.288477    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:38.288742 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:38.288617    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:38.398628 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:38.398597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r" event={"ID":"93354949-75ca-4479-a04b-1f74bdb0dd7a","Type":"ContainerStarted","Data":"72b03debac55a15230b6445c0327ddc0a58adbe39b0e9f4709105c738ceb3748"}
Jul 11 07:38:39.288412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.288375    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:39.288412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.288406    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:39.288676 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:39.288500    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:39.288676 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:39.288629    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:39.404603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.404441    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:38:39.405028 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.404996    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"7f31d341287b8b5206cb66cf59124f7b4602db5b69d72dd6a4ea2ee345792f02"}
Jul 11 07:38:39.669487 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.669447    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:39.670153 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.670132    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:39.683146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:39.683101    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plk5r" podStartSLOduration=5.357006038 podStartE2EDuration="25.683082304s" totalImagesPullingTime="20.326076266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:38.41360466 +0000 UTC m=+24.617023513" watchObservedRunningTime="2026-07-11 07:38:39.683082304 +0000 UTC m=+25.886501168"
Jul 11 07:38:40.289047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:40.289015    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:40.289223 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:40.289138    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:41.288854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.288672    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:41.288854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.288672    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:41.289618 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:41.289016    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:41.289618 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:41.289052    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:41.412112 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.412091    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:38:41.412429 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.412407    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"3f3ca08aa0a80efab5e1f8a753f231ddf33ea73289096214f429d0077664b716"}
Jul 11 07:38:41.412702 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.412661    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:41.412808 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.412731    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:41.412858 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.412850    2572 scope.go:117] "RemoveContainer" containerID="5a648a288701a9d906401c5d5774430daf26eeec532f90843532f0d1dafefc7f"
Jul 11 07:38:41.414795 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.414772    2572 generic.go:356] "Generic (PLEG): container finished" podID="453778fe-8de5-49f4-b6dd-54592e220864" containerID="6821681d17ba3bce0403172f9c95b52efb106212c7da0a8aa58c232ac58cb006" exitCode=0
Jul 11 07:38:41.414897 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.414805    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerDied","Data":"6821681d17ba3bce0403172f9c95b52efb106212c7da0a8aa58c232ac58cb006"}
Jul 11 07:38:41.429957 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.429937    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:41.430519 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.430503    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:41.932249 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.932215    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:41.932477 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.932374    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 11 07:38:41.932843 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:41.932821    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-crg4g"
Jul 11 07:38:42.288626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.288468    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:42.288840 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:42.288662    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:42.420588 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.420561    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:38:42.421023 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.420937    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" event={"ID":"a60fee7f-75a5-4e89-93f0-e8f5e571475a","Type":"ContainerStarted","Data":"bbaea62656fae23d43f249324972871b75493f1a656411aed7aa4f0583804d80"}
Jul 11 07:38:42.421023 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.421017    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 11 07:38:42.444903 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.444857    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" podStartSLOduration=9.814133273 podStartE2EDuration="28.444841318s" totalImagesPullingTime="18.630708045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:42.443661208 +0000 UTC m=+28.647080088" watchObservedRunningTime="2026-07-11 07:38:42.444841318 +0000 UTC m=+28.648260215"
Jul 11 07:38:42.529174 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.529134    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ngmk7"]
Jul 11 07:38:42.529277 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.529241    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:42.529373 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:42.529348    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:42.532255 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.532230    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pmtjg"]
Jul 11 07:38:42.532390 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.532348    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:42.532497 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:42.532443    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:42.532932 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.532914    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jnj7r"]
Jul 11 07:38:42.533035 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:42.533015    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:42.533134 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:42.533119    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:43.427726 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:43.427552    2572 generic.go:356] "Generic (PLEG): container finished" podID="453778fe-8de5-49f4-b6dd-54592e220864" containerID="6a9e05d064e46fcce076bc247aafef8a70f029f0b8a01b3c119038ec0f2a9e3e" exitCode=0
Jul 11 07:38:43.428094 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:43.427616    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerDied","Data":"6a9e05d064e46fcce076bc247aafef8a70f029f0b8a01b3c119038ec0f2a9e3e"}
Jul 11 07:38:43.428094 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:43.427904    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 11 07:38:44.288985 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:44.288956    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:44.289198 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:44.289044    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:44.289198 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:44.289065    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:44.289198 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:44.289087    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:44.289198 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:44.289160    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:44.289409 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:44.289217    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:44.959761 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:44.959730    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:45.038840 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.038807    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:45.259333 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.259242    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:38:45.259480 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.259439    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 11 07:38:45.269050 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.269013    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" podUID="a60fee7f-75a5-4e89-93f0-e8f5e571475a" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 11 07:38:45.276940 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.276913    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75" podUID="a60fee7f-75a5-4e89-93f0-e8f5e571475a" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 11 07:38:45.434470 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.434438    2572 generic.go:356] "Generic (PLEG): container finished" podID="453778fe-8de5-49f4-b6dd-54592e220864" containerID="278c94074c9bf79a719e172184137c5b6a3049e1e03a733c5d0c123e6252ad70" exitCode=0
Jul 11 07:38:45.434618 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:45.434500    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerDied","Data":"278c94074c9bf79a719e172184137c5b6a3049e1e03a733c5d0c123e6252ad70"}
Jul 11 07:38:46.288682 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:46.288650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:46.289176 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:46.288650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:46.289176 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:46.288754    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmtjg" podUID="ebe9b4a8-f4d9-4407-8874-767a013e4ab4"
Jul 11 07:38:46.289176 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:46.288845    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ngmk7" podUID="6b0e50ed-dc95-4271-96ec-3320f2ef2745"
Jul 11 07:38:46.289176 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:46.288654    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:46.289176 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:46.288951    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:38:47.905334 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:47.905301    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:47.905891 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:47.905471    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:47.905891 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:47.905557    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret podName:ebe9b4a8-f4d9-4407-8874-767a013e4ab4 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:19.905520905 +0000 UTC m=+66.108939770 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret") pod "global-pull-secret-syncer-pmtjg" (UID: "ebe9b4a8-f4d9-4407-8874-767a013e4ab4") : object "kube-system"/"original-pull-secret" not registered
Jul 11 07:38:48.006279 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.006250    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:48.006397 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.006287    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:48.006457 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.006396    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:48.006457 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.006411    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 07:38:48.006457 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.006427    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 07:38:48.006457 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.006440    2572 projected.go:196] Error preparing data for projected volume kube-api-access-mzw69 for pod openshift-network-diagnostics/network-check-target-ngmk7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:48.006647 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.006465    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:20.00644775 +0000 UTC m=+66.209866604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 07:38:48.006647 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.006502    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69 podName:6b0e50ed-dc95-4271-96ec-3320f2ef2745 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:20.00649232 +0000 UTC m=+66.209911161 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mzw69" (UniqueName: "kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69") pod "network-check-target-ngmk7" (UID: "6b0e50ed-dc95-4271-96ec-3320f2ef2745") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 07:38:48.124456 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.124429    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-181.ec2.internal" event="NodeReady"
Jul 11 07:38:48.124644 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.124581    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 11 07:38:48.161841 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.161772    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fbmfs"]
Jul 11 07:38:48.163597 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.163582    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.164747 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.164726    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r6rnj"]
Jul 11 07:38:48.165666 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.165644    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbf8m\""
Jul 11 07:38:48.165791 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.165770    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 11 07:38:48.165839 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.165774    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 11 07:38:48.166436 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.166418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.168540 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.168488    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 11 07:38:48.169024 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.168887    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-57t4x\""
Jul 11 07:38:48.169024 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.168931    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 11 07:38:48.169024 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.168990    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 11 07:38:48.173673 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.173627    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fbmfs"]
Jul 11 07:38:48.176291 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.176270    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6rnj"]
Jul 11 07:38:48.268940 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.268899    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:48.288767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.288742    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:38:48.288767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.288753    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:38:48.288767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.288766    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:38:48.291141 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.291050    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 11 07:38:48.291272 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.291143    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 11 07:38:48.291848 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.291429    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 11 07:38:48.291848 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.291461    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-92q58\""
Jul 11 07:38:48.291848 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.291627    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 11 07:38:48.291848 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.291662    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs4vf\""
Jul 11 07:38:48.308465 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.308442    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5r82\" (UniqueName: \"kubernetes.io/projected/62effbbf-9d06-4b9e-a9f0-95d23f91a8d8-kube-api-access-s5r82\") pod \"ingress-canary-r6rnj\" (UID: \"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8\") " pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.308632 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.308517    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.308632 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.308607    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b0367759-a5db-44e9-aa69-2baec901fd6a-config-volume\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.308736 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.308645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62effbbf-9d06-4b9e-a9f0-95d23f91a8d8-cert\") pod \"ingress-canary-r6rnj\" (UID: \"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8\") " pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.308736 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.308672    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0367759-a5db-44e9-aa69-2baec901fd6a-tmp-dir\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.308736 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.308696    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndjlg\" (UniqueName: \"kubernetes.io/projected/b0367759-a5db-44e9-aa69-2baec901fd6a-kube-api-access-ndjlg\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.409869 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.409833    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62effbbf-9d06-4b9e-a9f0-95d23f91a8d8-cert\") pod \"ingress-canary-r6rnj\" (UID: \"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8\") " pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.409869 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.409871    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0367759-a5db-44e9-aa69-2baec901fd6a-tmp-dir\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.410084 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.409896    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndjlg\" (UniqueName: \"kubernetes.io/projected/b0367759-a5db-44e9-aa69-2baec901fd6a-kube-api-access-ndjlg\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.410084 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.409955    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5r82\" (UniqueName: \"kubernetes.io/projected/62effbbf-9d06-4b9e-a9f0-95d23f91a8d8-kube-api-access-s5r82\") pod \"ingress-canary-r6rnj\" (UID: \"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8\") " pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.410084 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.410007    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.410084 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.410039    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b0367759-a5db-44e9-aa69-2baec901fd6a-config-volume\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.410272 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.410220    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0367759-a5db-44e9-aa69-2baec901fd6a-tmp-dir\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.410387 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.410366    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:38:48.410472 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.410428    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:38:48.910412685 +0000 UTC m=+35.113831535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:38:48.410686 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.410669    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b0367759-a5db-44e9-aa69-2baec901fd6a-config-volume\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.414200 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.414009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62effbbf-9d06-4b9e-a9f0-95d23f91a8d8-cert\") pod \"ingress-canary-r6rnj\" (UID: \"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8\") " pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.418001 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.417983    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndjlg\" (UniqueName: \"kubernetes.io/projected/b0367759-a5db-44e9-aa69-2baec901fd6a-kube-api-access-ndjlg\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.418426 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.418260    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5r82\" (UniqueName: \"kubernetes.io/projected/62effbbf-9d06-4b9e-a9f0-95d23f91a8d8-kube-api-access-s5r82\") pod \"ingress-canary-r6rnj\" (UID: \"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8\") " pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.482647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.482600    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6rnj"
Jul 11 07:38:48.624272 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.624204    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6rnj"]
Jul 11 07:38:48.630193 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:38:48.630159    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62effbbf_9d06_4b9e_a9f0_95d23f91a8d8.slice/crio-9b1102cd230071bc2fbac97e233bc99c1647c0e69ee165e3b79735d104efd59b WatchSource:0}: Error finding container 9b1102cd230071bc2fbac97e233bc99c1647c0e69ee165e3b79735d104efd59b: Status 404 returned error can't find the container with id 9b1102cd230071bc2fbac97e233bc99c1647c0e69ee165e3b79735d104efd59b
Jul 11 07:38:48.759247 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.759140    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:48.859841 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.859805    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:48.915154 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:48.915121    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:48.915720 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.915297    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:38:48.915720 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:48.915377    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:38:49.915357875 +0000 UTC m=+36.118776716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:38:49.444417 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:49.444381    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6rnj" event={"ID":"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8","Type":"ContainerStarted","Data":"9b1102cd230071bc2fbac97e233bc99c1647c0e69ee165e3b79735d104efd59b"}
Jul 11 07:38:49.513110 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:49.513079    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:49.613921 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:49.613888    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:49.921541 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:49.921498    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:49.921942 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:49.921653    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:38:49.921942 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:49.921731    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:38:51.921706468 +0000 UTC m=+38.125125315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:38:50.450136 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:50.450092    2572 generic.go:356] "Generic (PLEG): container finished" podID="453778fe-8de5-49f4-b6dd-54592e220864" containerID="a999dc1bc113bd9e503609975309c310c4c1ac87fa78336e54117ee587a03507" exitCode=0
Jul 11 07:38:50.450268 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:50.450147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerDied","Data":"a999dc1bc113bd9e503609975309c310c4c1ac87fa78336e54117ee587a03507"}
Jul 11 07:38:51.456504 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:51.456255    2572 generic.go:356] "Generic (PLEG): container finished" podID="453778fe-8de5-49f4-b6dd-54592e220864" containerID="711e27faa6c1abaf97e5b1fe038caa42d53b4ab80edd2ff0dc20214e398da478" exitCode=0
Jul 11 07:38:51.456905 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:51.456301    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerDied","Data":"711e27faa6c1abaf97e5b1fe038caa42d53b4ab80edd2ff0dc20214e398da478"}
Jul 11 07:38:51.940011 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:51.939988    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:51.940116 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:51.940093    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:38:51.940154 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:51.940136    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:38:55.940122946 +0000 UTC m=+42.143541787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:38:52.462397 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:52.462359    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lh56c" event={"ID":"453778fe-8de5-49f4-b6dd-54592e220864","Type":"ContainerStarted","Data":"6e1c84a07101f058ac377f77c41581317d3e64f916ae78accb3d60d7fc4b7ece"}
Jul 11 07:38:52.463766 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:52.463702    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6rnj" event={"ID":"62effbbf-9d06-4b9e-a9f0-95d23f91a8d8","Type":"ContainerStarted","Data":"a760a7380079f58301bb6b71419ce4c42f16f640bb60e12541af5341b21181c8"}
Jul 11 07:38:52.482297 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:52.482254    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lh56c" podStartSLOduration=9.176171089 podStartE2EDuration="38.482239799s" totalImagesPullingTime="29.30606871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:52.481093069 +0000 UTC m=+38.684511931" watchObservedRunningTime="2026-07-11 07:38:52.482239799 +0000 UTC m=+38.685658662"
Jul 11 07:38:52.494143 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:52.494087    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r6rnj" podStartSLOduration=1.142386721 podStartE2EDuration="4.494066528s" totalImagesPullingTime="3.351679807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:52.493689164 +0000 UTC m=+38.697108040" watchObservedRunningTime="2026-07-11 07:38:52.494066528 +0000 UTC m=+38.697485390"
Jul 11 07:38:52.708206 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:52.708175    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:52.810875 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:52.810844    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.013026 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.012991    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.111003 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.110919    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.211485 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.211452    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.263838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.263810    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.743908 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.743866    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.795762 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.795730    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:53.940596 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:53.940518    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:54.044084 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:54.044006    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:54.142732 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:54.142698    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:54.241974 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:54.241940    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:38:55.970272 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:55.970240    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:38:55.970749 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:55.970416    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:38:55.970749 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:38:55.970493    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:39:03.970472363 +0000 UTC m=+50.173891210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:38:58.315351 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:38:58.315292    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:04.024072 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:04.024037    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:39:04.024469 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:04.024141    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:39:04.024469 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:04.024189    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:39:20.024176881 +0000 UTC m=+66.227595722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:39:05.162588 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:05.162553    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:05.218425 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:05.218396    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:09.106235 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:09.106204    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:09.206203 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:09.206173    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:09.262932 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:09.262901    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:10.217326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:10.217298    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:10.261913 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:10.261877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:10.729566 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:10.729509    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:10.833207 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:10.833169    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:15.278657 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:15.278631    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8lm75"
Jul 11 07:39:15.425040 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:15.424988    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:15.524354 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:15.524330    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:19.925959 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:19.925922    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:39:19.928563 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:19.928547    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 11 07:39:19.939684 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:19.939654    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebe9b4a8-f4d9-4407-8874-767a013e4ab4-original-pull-secret\") pod \"global-pull-secret-syncer-pmtjg\" (UID: \"ebe9b4a8-f4d9-4407-8874-767a013e4ab4\") " pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:39:20.027040 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.027008    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:39:20.027040 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.027044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:39:20.027205 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.027065    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:39:20.027205 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:20.027153    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:39:20.027280 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:20.027215    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:39:52.027201304 +0000 UTC m=+98.230620149 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:39:20.029121 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.029104    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 11 07:39:20.029367 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.029353    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 11 07:39:20.037768 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:20.037754    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 11 07:39:20.037827 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:20.037806    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:40:24.037791158 +0000 UTC m=+130.241210003 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : secret "metrics-daemon-secret" not found
Jul 11 07:39:20.039419 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.039406    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 11 07:39:20.050569 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.050544    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzw69\" (UniqueName: \"kubernetes.io/projected/6b0e50ed-dc95-4271-96ec-3320f2ef2745-kube-api-access-mzw69\") pod \"network-check-target-ngmk7\" (UID: \"6b0e50ed-dc95-4271-96ec-3320f2ef2745\") " pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:39:20.100628 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.100595    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmtjg"
Jul 11 07:39:20.114507 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.114486    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-92q58\""
Jul 11 07:39:20.123284 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.123267    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:39:20.241748 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.241716    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pmtjg"]
Jul 11 07:39:20.245047 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:39:20.245025    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe9b4a8_f4d9_4407_8874_767a013e4ab4.slice/crio-525241deb49fee16d6ea07c6562d34814114dd6ba640654bc6ceb417124917b7 WatchSource:0}: Error finding container 525241deb49fee16d6ea07c6562d34814114dd6ba640654bc6ceb417124917b7: Status 404 returned error can't find the container with id 525241deb49fee16d6ea07c6562d34814114dd6ba640654bc6ceb417124917b7
Jul 11 07:39:20.264801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.264774    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ngmk7"]
Jul 11 07:39:20.268652 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:39:20.268631    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0e50ed_dc95_4271_96ec_3320f2ef2745.slice/crio-ff9d100666f7fe7d13842640bc12e31a835b273ab3e2e50008b0258d17fc3a9e WatchSource:0}: Error finding container ff9d100666f7fe7d13842640bc12e31a835b273ab3e2e50008b0258d17fc3a9e: Status 404 returned error can't find the container with id ff9d100666f7fe7d13842640bc12e31a835b273ab3e2e50008b0258d17fc3a9e
Jul 11 07:39:20.530004 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.529932    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ngmk7" event={"ID":"6b0e50ed-dc95-4271-96ec-3320f2ef2745","Type":"ContainerStarted","Data":"ff9d100666f7fe7d13842640bc12e31a835b273ab3e2e50008b0258d17fc3a9e"}
Jul 11 07:39:20.530973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:20.530945    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pmtjg" event={"ID":"ebe9b4a8-f4d9-4407-8874-767a013e4ab4","Type":"ContainerStarted","Data":"525241deb49fee16d6ea07c6562d34814114dd6ba640654bc6ceb417124917b7"}
Jul 11 07:39:25.546291 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:25.546258    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ngmk7" event={"ID":"6b0e50ed-dc95-4271-96ec-3320f2ef2745","Type":"ContainerStarted","Data":"e748cd2235ea112b82545e8e414b79b050e0e43cce8365f3a2d0ee081c5ad1d2"}
Jul 11 07:39:25.546753 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:25.546339    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:39:25.547698 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:25.547673    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pmtjg" event={"ID":"ebe9b4a8-f4d9-4407-8874-767a013e4ab4","Type":"ContainerStarted","Data":"39a117525fefb461cccc5b07a408a3e2d21eb733e76a8dd63a52670eeb8d6d75"}
Jul 11 07:39:25.559179 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:25.559141    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ngmk7" podStartSLOduration=67.244132641 podStartE2EDuration="1m11.55912895s" totalImagesPullingTime="4.314996309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:39:25.558955636 +0000 UTC m=+71.762374500" watchObservedRunningTime="2026-07-11 07:39:25.55912895 +0000 UTC m=+71.762547791"
Jul 11 07:39:31.102618 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:31.102585    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:31.204679 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:31.204650    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:31.259813 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:31.259778    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:39:52.034021 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:52.033885    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:39:52.034021 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:52.034004    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:39:52.034508 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:39:52.034065    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:40:56.034052256 +0000 UTC m=+162.237471098 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:39:56.551921 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:56.551890    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ngmk7"
Jul 11 07:39:56.565731 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:39:56.565686    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pmtjg" podStartSLOduration=98.223469344 podStartE2EDuration="1m42.565657115s" totalImagesPullingTime="4.342187771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:39:25.570876885 +0000 UTC m=+71.774295748" watchObservedRunningTime="2026-07-11 07:39:56.565657115 +0000 UTC m=+102.769075956"
Jul 11 07:40:24.042901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:40:24.042859    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:40:24.043366 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:40:24.043002    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 11 07:40:24.043366 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:40:24.043106    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:26.043080625 +0000 UTC m=+252.246499469 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : secret "metrics-daemon-secret" not found
Jul 11 07:40:51.177392 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:40:51.177349    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fbmfs" podUID="b0367759-a5db-44e9-aa69-2baec901fd6a"
Jul 11 07:40:51.306643 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:40:51.306593    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:40:51.746185 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:40:51.746147    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:40:56.046903 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:40:56.046866    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:40:56.047296 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:40:56.046977    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 07:40:56.047296 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:40:56.047025    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls podName:b0367759-a5db-44e9-aa69-2baec901fd6a nodeName:}" failed. No retries permitted until 2026-07-11 07:42:58.047012394 +0000 UTC m=+284.250431239 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls") pod "dns-default-fbmfs" (UID: "b0367759-a5db-44e9-aa69-2baec901fd6a") : secret "dns-default-metrics-tls" not found
Jul 11 07:41:03.288316 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:41:03.288281    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:41:46.691608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:41:46.691576    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:12.349850 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.349742    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"]
Jul 11 07:42:12.351688 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.351672    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"]
Jul 11 07:42:12.351822 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.351805    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"
Jul 11 07:42:12.353290 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.353263    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.353969 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.353938    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f88d7c45d-dh675"]
Jul 11 07:42:12.354078 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.353945    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 11 07:42:12.354140 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.354122    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-phptq\""
Jul 11 07:42:12.354381 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.354361    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 11 07:42:12.355546 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.355516    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.358749 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.358683    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 11 07:42:12.358816 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.358791    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 11 07:42:12.359047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.358886    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 11 07:42:12.359047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.358912    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 11 07:42:12.359047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.358959    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sf5g5\""
Jul 11 07:42:12.359047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.358975    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 11 07:42:12.359047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.359024    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4gzdp\""
Jul 11 07:42:12.359292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.359179    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 11 07:42:12.359292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.359254    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 11 07:42:12.364441 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.364423    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 11 07:42:12.366453 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.366434    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"]
Jul 11 07:42:12.376034 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.376011    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"]
Jul 11 07:42:12.376819 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.376800    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f88d7c45d-dh675"]
Jul 11 07:42:12.445954 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.445920    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"]
Jul 11 07:42:12.447782 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.447766    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.448214 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448188    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tdbk\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-kube-api-access-4tdbk\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448317 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448244    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-certificates\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448317 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448278    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxpll\" (UniqueName: \"kubernetes.io/projected/1c488d3c-413b-4146-a23a-8403f16df1e9-kube-api-access-nxpll\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.448317 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-trusted-ca\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448433 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c488d3c-413b-4146-a23a-8403f16df1e9-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.448433 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448363    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-installation-pull-secrets\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448539 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448433    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.448539 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448479    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a77e19c-65fc-4bb1-b601-895a19b8c967-ca-trust-extracted\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448539 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448514    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-bound-sa-token\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448692 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448676    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86p9x\" (UniqueName: \"kubernetes.io/projected/c6b3de6a-29b0-4b25-81cb-22080bbfd51f-kube-api-access-86p9x\") pod \"volume-data-source-validator-78495597f6-vrnd7\" (UID: \"c6b3de6a-29b0-4b25-81cb-22080bbfd51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"
Jul 11 07:42:12.448744 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-image-registry-private-configuration\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.448744 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.448714    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.450016 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.450000    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 11 07:42:12.450237 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.450213    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8hjqb\""
Jul 11 07:42:12.450358 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.450341    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 11 07:42:12.450425 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.450388    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 11 07:42:12.450597 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.450584    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 11 07:42:12.460503 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.460478    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"]
Jul 11 07:42:12.549565 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdbk\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-kube-api-access-4tdbk\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.549565 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549569    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-certificates\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxpll\" (UniqueName: \"kubernetes.io/projected/1c488d3c-413b-4146-a23a-8403f16df1e9-kube-api-access-nxpll\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-trusted-ca\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c488d3c-413b-4146-a23a-8403f16df1e9-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549649    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-installation-pull-secrets\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549669    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b996739e-74c6-4e14-a7f6-20594f36940d-config\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549696    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpw5x\" (UniqueName: \"kubernetes.io/projected/b996739e-74c6-4e14-a7f6-20594f36940d-kube-api-access-fpw5x\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a77e19c-65fc-4bb1-b601-895a19b8c967-ca-trust-extracted\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549797    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-bound-sa-token\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549825    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b996739e-74c6-4e14-a7f6-20594f36940d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.549898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.549895    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86p9x\" (UniqueName: \"kubernetes.io/projected/c6b3de6a-29b0-4b25-81cb-22080bbfd51f-kube-api-access-86p9x\") pod \"volume-data-source-validator-78495597f6-vrnd7\" (UID: \"c6b3de6a-29b0-4b25-81cb-22080bbfd51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:12.549898    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.550017    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-image-registry-private-configuration\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.550044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:12.550146    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:12.550158    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f88d7c45d-dh675: secret "image-registry-tls" not found
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:12.550191    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls podName:1c488d3c-413b-4146-a23a-8403f16df1e9 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:13.050172515 +0000 UTC m=+239.253591360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7t75d" (UID: "1c488d3c-413b-4146-a23a-8403f16df1e9") : secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.550200    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a77e19c-65fc-4bb1-b601-895a19b8c967-ca-trust-extracted\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:12.550209    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls podName:3a77e19c-65fc-4bb1-b601-895a19b8c967 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:13.050199549 +0000 UTC m=+239.253618396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls") pod "image-registry-7f88d7c45d-dh675" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967") : secret "image-registry-tls" not found
Jul 11 07:42:12.550510 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.550430    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c488d3c-413b-4146-a23a-8403f16df1e9-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.550824 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.550646    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-trusted-ca\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.550824 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.550715    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-certificates\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.552185 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.552164    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-installation-pull-secrets\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.552278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.552204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-image-registry-private-configuration\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.558484 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.558462    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86p9x\" (UniqueName: \"kubernetes.io/projected/c6b3de6a-29b0-4b25-81cb-22080bbfd51f-kube-api-access-86p9x\") pod \"volume-data-source-validator-78495597f6-vrnd7\" (UID: \"c6b3de6a-29b0-4b25-81cb-22080bbfd51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"
Jul 11 07:42:12.558755 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.558736    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxpll\" (UniqueName: \"kubernetes.io/projected/1c488d3c-413b-4146-a23a-8403f16df1e9-kube-api-access-nxpll\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:12.559212 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.559195    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-bound-sa-token\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.559321 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.559305    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tdbk\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-kube-api-access-4tdbk\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:12.566949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.566924    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:12.615193 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.615129    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:12.650490 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.650447    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b996739e-74c6-4e14-a7f6-20594f36940d-config\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.650490 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.650494    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpw5x\" (UniqueName: \"kubernetes.io/projected/b996739e-74c6-4e14-a7f6-20594f36940d-kube-api-access-fpw5x\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.650677 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.650554    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b996739e-74c6-4e14-a7f6-20594f36940d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.651078 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.651054    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b996739e-74c6-4e14-a7f6-20594f36940d-config\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.652638 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.652618    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b996739e-74c6-4e14-a7f6-20594f36940d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.658236 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.658206    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpw5x\" (UniqueName: \"kubernetes.io/projected/b996739e-74c6-4e14-a7f6-20594f36940d-kube-api-access-fpw5x\") pod \"service-ca-operator-7695fc6c8c-lzt6b\" (UID: \"b996739e-74c6-4e14-a7f6-20594f36940d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.661985 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.661969    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"
Jul 11 07:42:12.756058 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.756026    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"
Jul 11 07:42:12.787579 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.785617    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7"]
Jul 11 07:42:12.791759 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:12.791733    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b3de6a_29b0_4b25_81cb_22080bbfd51f.slice/crio-fb10def36f4c55372abf3fa99fa73d564e9a97b113dd9c9c7059625da3a72602 WatchSource:0}: Error finding container fb10def36f4c55372abf3fa99fa73d564e9a97b113dd9c9c7059625da3a72602: Status 404 returned error can't find the container with id fb10def36f4c55372abf3fa99fa73d564e9a97b113dd9c9c7059625da3a72602
Jul 11 07:42:12.881943 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.881851    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b"]
Jul 11 07:42:12.885583 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:12.885562    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb996739e_74c6_4e14_a7f6_20594f36940d.slice/crio-7eaae22522154b61699f7569145e3b997c925d2395074eb1208658a479d9b0df WatchSource:0}: Error finding container 7eaae22522154b61699f7569145e3b997c925d2395074eb1208658a479d9b0df: Status 404 returned error can't find the container with id 7eaae22522154b61699f7569145e3b997c925d2395074eb1208658a479d9b0df
Jul 11 07:42:12.934518 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.934483    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b" event={"ID":"b996739e-74c6-4e14-a7f6-20594f36940d","Type":"ContainerStarted","Data":"7eaae22522154b61699f7569145e3b997c925d2395074eb1208658a479d9b0df"}
Jul 11 07:42:12.935480 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:12.935462    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7" event={"ID":"c6b3de6a-29b0-4b25-81cb-22080bbfd51f","Type":"ContainerStarted","Data":"fb10def36f4c55372abf3fa99fa73d564e9a97b113dd9c9c7059625da3a72602"}
Jul 11 07:42:13.053322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:13.053287    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:13.053472 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:13.053371    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:13.053472 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:13.053441    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 07:42:13.053472 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:13.053459    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f88d7c45d-dh675: secret "image-registry-tls" not found
Jul 11 07:42:13.053472 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:13.053463    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:13.053669 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:13.053511    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls podName:3a77e19c-65fc-4bb1-b601-895a19b8c967 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:14.053495549 +0000 UTC m=+240.256914390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls") pod "image-registry-7f88d7c45d-dh675" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967") : secret "image-registry-tls" not found
Jul 11 07:42:13.053669 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:13.053543    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls podName:1c488d3c-413b-4146-a23a-8403f16df1e9 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:14.053518642 +0000 UTC m=+240.256937483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7t75d" (UID: "1c488d3c-413b-4146-a23a-8403f16df1e9") : secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:14.062410 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:14.062374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:14.062875 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:14.062467    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:14.062875 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:14.062503    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:14.062875 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:14.062607    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls podName:1c488d3c-413b-4146-a23a-8403f16df1e9 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:16.06257536 +0000 UTC m=+242.265994218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7t75d" (UID: "1c488d3c-413b-4146-a23a-8403f16df1e9") : secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:14.062875 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:14.062656    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 07:42:14.062875 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:14.062673    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f88d7c45d-dh675: secret "image-registry-tls" not found
Jul 11 07:42:14.062875 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:14.062722    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls podName:3a77e19c-65fc-4bb1-b601-895a19b8c967 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:16.062707723 +0000 UTC m=+242.266126565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls") pod "image-registry-7f88d7c45d-dh675" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967") : secret "image-registry-tls" not found
Jul 11 07:42:14.943013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:14.942976    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7" event={"ID":"c6b3de6a-29b0-4b25-81cb-22080bbfd51f","Type":"ContainerStarted","Data":"ea29ab1b126560908fa674c51205c3738c04008ac2edeaf129ff331984c45df0"}
Jul 11 07:42:14.944355 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:14.944329    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b" event={"ID":"b996739e-74c6-4e14-a7f6-20594f36940d","Type":"ContainerStarted","Data":"0cf2dc797a1729518ab63debd2bae12352bddda1739f9d188368d8d4e15318fa"}
Jul 11 07:42:14.958658 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:14.958613    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vrnd7" podStartSLOduration=1.611399718 podStartE2EDuration="2.958596098s" totalImagesPullingTime="1.34719638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:14.958215382 +0000 UTC m=+241.161634245" watchObservedRunningTime="2026-07-11 07:42:14.958596098 +0000 UTC m=+241.162014960"
Jul 11 07:42:14.973316 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:14.973260    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b" podStartSLOduration=1.030687664 podStartE2EDuration="2.973240262s" totalImagesPullingTime="1.942552598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:14.973025753 +0000 UTC m=+241.176444616" watchObservedRunningTime="2026-07-11 07:42:14.973240262 +0000 UTC m=+241.176659125"
Jul 11 07:42:15.030228 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:15.030197    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:16.012768 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.012740    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:16.078731 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.078689    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:16.078918 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.078803    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:16.078918 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:16.078832    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 07:42:16.078918 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:16.078850    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f88d7c45d-dh675: secret "image-registry-tls" not found
Jul 11 07:42:16.078918 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:16.078895    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls podName:3a77e19c-65fc-4bb1-b601-895a19b8c967 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.078880811 +0000 UTC m=+246.282299652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls") pod "image-registry-7f88d7c45d-dh675" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967") : secret "image-registry-tls" not found
Jul 11 07:42:16.078918 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:16.078914    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:16.079115 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:16.078965    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls podName:1c488d3c-413b-4146-a23a-8403f16df1e9 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.078949159 +0000 UTC m=+246.282368004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7t75d" (UID: "1c488d3c-413b-4146-a23a-8403f16df1e9") : secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:16.111961 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.111930    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:16.486692 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.486657    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:16.542520 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.542481    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:16.841676 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.841646    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:16.942187 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:16.942152    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:17.123788 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.123707    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:17.145172 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.145145    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"]
Jul 11 07:42:17.147182 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.147161    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"
Jul 11 07:42:17.149472 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.149448    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 11 07:42:17.149472 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.149477    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t57tl\""
Jul 11 07:42:17.149642 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.149567    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 11 07:42:17.158941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.158910    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"]
Jul 11 07:42:17.189672 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.189647    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc6c4\" (UniqueName: \"kubernetes.io/projected/8652c53f-c34f-4274-b2a9-6782aea58a8a-kube-api-access-cc6c4\") pod \"migrator-7f59b54f6c-tjdnr\" (UID: \"8652c53f-c34f-4274-b2a9-6782aea58a8a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"
Jul 11 07:42:17.204971 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.204943    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:17.220922 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.220890    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-h4nnq"]
Jul 11 07:42:17.222852 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.222834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.225031 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.225007    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 11 07:42:17.225031 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.225028    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 11 07:42:17.225182 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.225145    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 11 07:42:17.225332 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.225317    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mjxk7\""
Jul 11 07:42:17.225390 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.225375    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 11 07:42:17.232852 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.232829    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h4nnq"]
Jul 11 07:42:17.290771 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.290741    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.290771 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.290782    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc6c4\" (UniqueName: \"kubernetes.io/projected/8652c53f-c34f-4274-b2a9-6782aea58a8a-kube-api-access-cc6c4\") pod \"migrator-7f59b54f6c-tjdnr\" (UID: \"8652c53f-c34f-4274-b2a9-6782aea58a8a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"
Jul 11 07:42:17.290975 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.290845    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhss7\" (UniqueName: \"kubernetes.io/projected/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-api-access-rhss7\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.290975 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.290875    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cba86ef-a175-40d8-ae06-54eed94ff4a5-data-volume\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.290975 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.290923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cba86ef-a175-40d8-ae06-54eed94ff4a5-crio-socket\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.291060 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.290996    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.297629 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.297603    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc6c4\" (UniqueName: \"kubernetes.io/projected/8652c53f-c34f-4274-b2a9-6782aea58a8a-kube-api-access-cc6c4\") pod \"migrator-7f59b54f6c-tjdnr\" (UID: \"8652c53f-c34f-4274-b2a9-6782aea58a8a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"
Jul 11 07:42:17.318724 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.318699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:17.391838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.391768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.391838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.391808    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.391838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.391834    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhss7\" (UniqueName: \"kubernetes.io/projected/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-api-access-rhss7\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.391858    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cba86ef-a175-40d8-ae06-54eed94ff4a5-data-volume\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.391884    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cba86ef-a175-40d8-ae06-54eed94ff4a5-crio-socket\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.391899    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.391967    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:17.891947741 +0000 UTC m=+244.095366598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : configmap "kube-rbac-proxy" not found
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.391974    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.392008    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cba86ef-a175-40d8-ae06-54eed94ff4a5-crio-socket\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.392064 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.392025    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:17.892009612 +0000 UTC m=+244.095428477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:17.392391 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.392182    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cba86ef-a175-40d8-ae06-54eed94ff4a5-data-volume\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.399225 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.399203    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhss7\" (UniqueName: \"kubernetes.io/projected/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-api-access-rhss7\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.455355 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.455324    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"
Jul 11 07:42:17.576152 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.576122    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr"]
Jul 11 07:42:17.579155 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:17.579125    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8652c53f_c34f_4274_b2a9_6782aea58a8a.slice/crio-1ef96bb6aa13795498741c2b9262ed7fcd0e2e2e528562575f15f093cf61a0df WatchSource:0}: Error finding container 1ef96bb6aa13795498741c2b9262ed7fcd0e2e2e528562575f15f093cf61a0df: Status 404 returned error can't find the container with id 1ef96bb6aa13795498741c2b9262ed7fcd0e2e2e528562575f15f093cf61a0df
Jul 11 07:42:17.895384 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.895343    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.895384 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.895393    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:17.895681 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.895487    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 07:42:17.895681 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.895494    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:17.895681 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.895560    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:18.895542463 +0000 UTC m=+245.098961319 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : configmap "kube-rbac-proxy" not found
Jul 11 07:42:17.895681 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:17.895577    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:18.89556863 +0000 UTC m=+245.098987471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:17.953631 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:17.953597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr" event={"ID":"8652c53f-c34f-4274-b2a9-6782aea58a8a","Type":"ContainerStarted","Data":"1ef96bb6aa13795498741c2b9262ed7fcd0e2e2e528562575f15f093cf61a0df"}
Jul 11 07:42:18.355801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.355768    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-wqs9v"]
Jul 11 07:42:18.357926 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.357908    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.360332 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.360302    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 11 07:42:18.360439 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.360375    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 11 07:42:18.360439 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.360400    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 11 07:42:18.361244 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.361223    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 11 07:42:18.361244 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.361238    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4d68g\""
Jul 11 07:42:18.367152 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.367133    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-wqs9v"]
Jul 11 07:42:18.384599 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.384575    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:18.399486 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.399466    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lstwd\" (UniqueName: \"kubernetes.io/projected/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-kube-api-access-lstwd\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.399602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.399538    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-signing-key\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.399640 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.399598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-signing-cabundle\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.500812 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.500778    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-signing-key\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.500976 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.500828    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-signing-cabundle\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.500976 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.500898    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lstwd\" (UniqueName: \"kubernetes.io/projected/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-kube-api-access-lstwd\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.501472 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.501452    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-signing-cabundle\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.503038 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.503021    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-signing-key\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.509119 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.509091    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lstwd\" (UniqueName: \"kubernetes.io/projected/efd6e4b6-7310-42ab-b9db-bf16b38a4a30-kube-api-access-lstwd\") pod \"service-ca-79f68b9f4f-wqs9v\" (UID: \"efd6e4b6-7310-42ab-b9db-bf16b38a4a30\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.667695 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.667609    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v"
Jul 11 07:42:18.848296 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.848271    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-wqs9v"]
Jul 11 07:42:18.855620 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:18.855595    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd6e4b6_7310_42ab_b9db_bf16b38a4a30.slice/crio-6c88fbb826a1c657c1442bb6a51b257ea0e000ce61b7f25eacad3975915fdf75 WatchSource:0}: Error finding container 6c88fbb826a1c657c1442bb6a51b257ea0e000ce61b7f25eacad3975915fdf75: Status 404 returned error can't find the container with id 6c88fbb826a1c657c1442bb6a51b257ea0e000ce61b7f25eacad3975915fdf75
Jul 11 07:42:18.904670 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.904646    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:18.904756 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.904694    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:18.904794 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:18.904780    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 07:42:18.904830 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:18.904822    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.904809888 +0000 UTC m=+247.108228730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : configmap "kube-rbac-proxy" not found
Jul 11 07:42:18.904872 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:18.904783    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:18.904872 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:18.904854    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.904845478 +0000 UTC m=+247.108264320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:18.959484 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.959399    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v" event={"ID":"efd6e4b6-7310-42ab-b9db-bf16b38a4a30","Type":"ContainerStarted","Data":"7454b9bd632a0e15b22b6883c752516a219442b167efc903c9ed82d645a5534b"}
Jul 11 07:42:18.959484 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.959450    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v" event={"ID":"efd6e4b6-7310-42ab-b9db-bf16b38a4a30","Type":"ContainerStarted","Data":"6c88fbb826a1c657c1442bb6a51b257ea0e000ce61b7f25eacad3975915fdf75"}
Jul 11 07:42:18.961142 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.961104    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr" event={"ID":"8652c53f-c34f-4274-b2a9-6782aea58a8a","Type":"ContainerStarted","Data":"204db005a1d7813e4ef18ddea98bf740d8e8bb3f31f70da589c90d67f8083206"}
Jul 11 07:42:18.961271 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.961155    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr" event={"ID":"8652c53f-c34f-4274-b2a9-6782aea58a8a","Type":"ContainerStarted","Data":"5dc96e5e9a826991f10aa98a9f909793d0d47f62a91f33b4939841abface88ce"}
Jul 11 07:42:18.973867 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.973816    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-wqs9v" podStartSLOduration=0.97379721 podStartE2EDuration="973.79721ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:18.973584994 +0000 UTC m=+245.177003867" watchObservedRunningTime="2026-07-11 07:42:18.97379721 +0000 UTC m=+245.177216074"
Jul 11 07:42:18.988563 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:18.988492    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tjdnr" podStartSLOduration=0.807453151 podStartE2EDuration="1.988480284s" totalImagesPullingTime="1.181027133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:18.987955447 +0000 UTC m=+245.191374328" watchObservedRunningTime="2026-07-11 07:42:18.988480284 +0000 UTC m=+245.191899146"
Jul 11 07:42:19.214135 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:19.214041    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:19.768820 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:19.768782    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.004593 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.004566    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jt4gq_547f78c4-01e1-4826-8413-0685d61c4f36/dns-node-resolver/0.log"
Jul 11 07:42:20.114829 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.114791    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:20.115000 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.114840    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:20.115000 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.114952    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:20.115131 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.115042    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls podName:1c488d3c-413b-4146-a23a-8403f16df1e9 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:28.115020471 +0000 UTC m=+254.318439324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7t75d" (UID: "1c488d3c-413b-4146-a23a-8403f16df1e9") : secret "cluster-monitoring-operator-tls" not found
Jul 11 07:42:20.115228 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.114959    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 07:42:20.115315 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.115295    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f88d7c45d-dh675: secret "image-registry-tls" not found
Jul 11 07:42:20.115458 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.115369    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls podName:3a77e19c-65fc-4bb1-b601-895a19b8c967 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:28.115353162 +0000 UTC m=+254.318772007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls") pod "image-registry-7f88d7c45d-dh675" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967") : secret "image-registry-tls" not found
Jul 11 07:42:20.201488 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.201453    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.311151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.311111    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.408290 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.408204    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.508626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.508580    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.608405 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.608371    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.708936 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.708861    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:20.921954 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.921921    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:20.922309 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:20.921972    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:20.922309 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.922081    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:20.922309 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.922138    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:24.922124289 +0000 UTC m=+251.125543136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:20.922309 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.922083    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 07:42:20.922309 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:20.922213    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:24.922202062 +0000 UTC m=+251.125620919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : configmap "kube-rbac-proxy" not found
Jul 11 07:42:21.005242 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:21.005174    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9gcz_fde0f7b1-e969-480a-8209-31641ab1966e/node-ca/0.log"
Jul 11 07:42:21.607981 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:21.607940    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:21.800453 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:21.800421    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:21.805634 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:21.805610    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6rnj_62effbbf-9d06-4b9e-a9f0-95d23f91a8d8/serve-healthcheck-canary/0.log"
Jul 11 07:42:21.998278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:21.998183    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:22.195840 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:22.195806    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:22.204808 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:22.204776    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-tjdnr_8652c53f-c34f-4274-b2a9-6782aea58a8a/migrator/0.log"
Jul 11 07:42:22.390782 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:22.390751    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:22.405573 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:22.405546    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-tjdnr_8652c53f-c34f-4274-b2a9-6782aea58a8a/graceful-termination/0.log"
Jul 11 07:42:22.596668 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:22.596630    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:23.401774 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:23.401740    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:23.591801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:23.591765    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:23.692791 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:23.692697    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:23.793374 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:23.793340    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:24.192397 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:24.192359    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:24.397498 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:24.397456    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:24.793719 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:24.793681    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:24.955502 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:24.955471    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:24.955678 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:24.955516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:24.955678 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:24.955611    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 07:42:24.955678 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:24.955646    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:24.955678 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:24.955659    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:32.955645754 +0000 UTC m=+259.159064598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : configmap "kube-rbac-proxy" not found
Jul 11 07:42:24.955828 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:24.955702    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls podName:8cba86ef-a175-40d8-ae06-54eed94ff4a5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:32.955685419 +0000 UTC m=+259.159104267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4nnq" (UID: "8cba86ef-a175-40d8-ae06-54eed94ff4a5") : secret "insights-runtime-extractor-tls" not found
Jul 11 07:42:25.593927 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:25.593888    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:25.796421 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:25.796385    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:25.954948 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:25.954856    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:26.069810 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:26.069775    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:42:26.069989 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:26.069924    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 11 07:42:26.070047 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:26.069991    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs podName:29f420da-3869-4a78-97a2-c4e78477cfb2 nodeName:}" failed. No retries permitted until 2026-07-11 07:44:28.069976528 +0000 UTC m=+374.273395370 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs") pod "network-metrics-daemon-jnj7r" (UID: "29f420da-3869-4a78-97a2-c4e78477cfb2") : secret "metrics-daemon-secret" not found
Jul 11 07:42:26.198352 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:26.198315    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:26.300203 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:26.300178    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:26.400017 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:26.399986    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:26.802481 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:26.802442    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:27.390148 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:27.390111    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:27.791794 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:27.791697    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:28.186160 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.186125    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:28.186160 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.186162    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:28.188501 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.188481    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"image-registry-7f88d7c45d-dh675\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") " pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:28.188568 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.188521    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c488d3c-413b-4146-a23a-8403f16df1e9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7t75d\" (UID: \"1c488d3c-413b-4146-a23a-8403f16df1e9\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:28.192178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.192150    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:28.270871 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.270845    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sf5g5\""
Jul 11 07:42:28.275344 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.275325    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4gzdp\""
Jul 11 07:42:28.279043 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.279031    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"
Jul 11 07:42:28.283836 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.283824    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:28.416044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.416015    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d"]
Jul 11 07:42:28.421516 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:28.421491    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c488d3c_413b_4146_a23a_8403f16df1e9.slice/crio-d2058cc0f5e34db70c121356a3292682c224ecf5cc53595197d10204d78c614d WatchSource:0}: Error finding container d2058cc0f5e34db70c121356a3292682c224ecf5cc53595197d10204d78c614d: Status 404 returned error can't find the container with id d2058cc0f5e34db70c121356a3292682c224ecf5cc53595197d10204d78c614d
Jul 11 07:42:28.430093 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.430069    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f88d7c45d-dh675"]
Jul 11 07:42:28.433119 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:28.433091    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a77e19c_65fc_4bb1_b601_895a19b8c967.slice/crio-794eb4aede19cb074e38729cdfb675a768716b71c3cb21848c54a7a56b9c9181 WatchSource:0}: Error finding container 794eb4aede19cb074e38729cdfb675a768716b71c3cb21848c54a7a56b9c9181: Status 404 returned error can't find the container with id 794eb4aede19cb074e38729cdfb675a768716b71c3cb21848c54a7a56b9c9181
Jul 11 07:42:28.588122 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.588084    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:28.792546 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.792448    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:28.994197 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.994159    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" event={"ID":"3a77e19c-65fc-4bb1-b601-895a19b8c967","Type":"ContainerStarted","Data":"6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87"}
Jul 11 07:42:28.994383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.994218    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" event={"ID":"3a77e19c-65fc-4bb1-b601-895a19b8c967","Type":"ContainerStarted","Data":"794eb4aede19cb074e38729cdfb675a768716b71c3cb21848c54a7a56b9c9181"}
Jul 11 07:42:28.994383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.994245    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:42:28.995444 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:28.995418    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d" event={"ID":"1c488d3c-413b-4146-a23a-8403f16df1e9","Type":"ContainerStarted","Data":"d2058cc0f5e34db70c121356a3292682c224ecf5cc53595197d10204d78c614d"}
Jul 11 07:42:29.013048 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:29.012999    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" podStartSLOduration=17.012982521 podStartE2EDuration="17.012982521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:29.0117404 +0000 UTC m=+255.215159263" watchObservedRunningTime="2026-07-11 07:42:29.012982521 +0000 UTC m=+255.216401380"
Jul 11 07:42:29.391074 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:29.391041    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:29.760745 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:29.760653    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:29.857502 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:29.857466    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:29.988692 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:29.988656    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:30.189021 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:30.188980    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:30.535053 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:30.534965    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:30.634981 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:30.634927    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:30.735236 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:30.735197    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:31.003630 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:31.003597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d" event={"ID":"1c488d3c-413b-4146-a23a-8403f16df1e9","Type":"ContainerStarted","Data":"f6fe2ba3717fe1c745148c5bc0f2773e4b8254a0394f850a0ab52e9c1a143e51"}
Jul 11 07:42:31.019729 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:31.019671    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7t75d" podStartSLOduration=17.204696284 podStartE2EDuration="19.019650898s" totalImagesPullingTime="1.814954614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:31.017496555 +0000 UTC m=+257.220915430" watchObservedRunningTime="2026-07-11 07:42:31.019650898 +0000 UTC m=+257.223069764"
Jul 11 07:42:31.232536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:31.232485    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:33.024414 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:33.024367    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:33.024832 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:33.024455    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:33.024903 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:33.024883    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba86ef-a175-40d8-ae06-54eed94ff4a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:33.026669 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:33.026649    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba86ef-a175-40d8-ae06-54eed94ff4a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4nnq\" (UID: \"8cba86ef-a175-40d8-ae06-54eed94ff4a5\") " pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:33.131004 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:33.130953    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h4nnq"
Jul 11 07:42:33.255415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:33.255382    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h4nnq"]
Jul 11 07:42:33.258922 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:33.258896    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cba86ef_a175_40d8_ae06_54eed94ff4a5.slice/crio-3818c732954605638991c5e9cc6ac921cd33ca899dc6f78ddab07346b7d593a0 WatchSource:0}: Error finding container 3818c732954605638991c5e9cc6ac921cd33ca899dc6f78ddab07346b7d593a0: Status 404 returned error can't find the container with id 3818c732954605638991c5e9cc6ac921cd33ca899dc6f78ddab07346b7d593a0
Jul 11 07:42:34.016860 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:34.016801    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4nnq" event={"ID":"8cba86ef-a175-40d8-ae06-54eed94ff4a5","Type":"ContainerStarted","Data":"3818c732954605638991c5e9cc6ac921cd33ca899dc6f78ddab07346b7d593a0"}
Jul 11 07:42:37.027347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:37.027314    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4nnq" event={"ID":"8cba86ef-a175-40d8-ae06-54eed94ff4a5","Type":"ContainerStarted","Data":"46c2eedc39488acecd804b0505b0ef13e847e9a2667519b0abca5d9de7ceb633"}
Jul 11 07:42:37.627985 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:37.627945    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:37.728020 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:37.727992    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:37.780034 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:37.779998    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:37.923909 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:37.921652    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.031942 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.031910    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4nnq" event={"ID":"8cba86ef-a175-40d8-ae06-54eed94ff4a5","Type":"ContainerStarted","Data":"dfed7bd2ae4c57624cd1b21933de2dc564985df0dad2afa384bd208a5ae85fd8"}
Jul 11 07:42:38.107026 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.106981    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.301613 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.301579    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.403067 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.403020    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.699039 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.698950    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.803033 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.802982    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.906412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.906348    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:38.919959 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.919929    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"]
Jul 11 07:42:38.923019 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.922995    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:38.925379 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.925360    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 11 07:42:38.925916 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.925888    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-228sh\""
Jul 11 07:42:38.932869 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.932836    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"]
Jul 11 07:42:38.939492 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.939470    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-nmbc4"]
Jul 11 07:42:38.942535 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.942507    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:38.944897 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.944876    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kn8q6\""
Jul 11 07:42:38.944995 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.944904    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 11 07:42:38.944995 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.944926    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 11 07:42:38.950452 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.950378    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f88d7c45d-dh675"]
Jul 11 07:42:38.955315 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:38.955256    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-nmbc4"]
Jul 11 07:42:39.013958 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.013919    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:39.074831 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.074797    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwpsh\" (UniqueName: \"kubernetes.io/projected/6be5391d-fe71-4a51-b7e0-79f9c31df6ff-kube-api-access-hwpsh\") pod \"downloads-68c9cbb497-nmbc4\" (UID: \"6be5391d-fe71-4a51-b7e0-79f9c31df6ff\") " pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:39.075275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.074882    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e284dcd0-1354-43df-88a8-a7b6a1e8c6de-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-2dkrv\" (UID: \"e284dcd0-1354-43df-88a8-a7b6a1e8c6de\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:39.114271 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.114192    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:39.176085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.176047    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e284dcd0-1354-43df-88a8-a7b6a1e8c6de-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-2dkrv\" (UID: \"e284dcd0-1354-43df-88a8-a7b6a1e8c6de\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:39.176266 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.176122    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwpsh\" (UniqueName: \"kubernetes.io/projected/6be5391d-fe71-4a51-b7e0-79f9c31df6ff-kube-api-access-hwpsh\") pod \"downloads-68c9cbb497-nmbc4\" (UID: \"6be5391d-fe71-4a51-b7e0-79f9c31df6ff\") " pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:39.178656 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.178635    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e284dcd0-1354-43df-88a8-a7b6a1e8c6de-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-2dkrv\" (UID: \"e284dcd0-1354-43df-88a8-a7b6a1e8c6de\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:39.187293 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.187268    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwpsh\" (UniqueName: \"kubernetes.io/projected/6be5391d-fe71-4a51-b7e0-79f9c31df6ff-kube-api-access-hwpsh\") pod \"downloads-68c9cbb497-nmbc4\" (UID: \"6be5391d-fe71-4a51-b7e0-79f9c31df6ff\") " pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:39.232756 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.232678    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:39.254520 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.254497    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:39.318551 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.318497    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:39.509308 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:39.509229    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:40.141905 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:40.141781    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:40.154233 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:40.154119    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-nmbc4"]
Jul 11 07:42:40.161456 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:40.161426    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be5391d_fe71_4a51_b7e0_79f9c31df6ff.slice/crio-4b45081f74fcae023b539de5a9ca459f275d94d984f05cf975e5ca1283fb738d WatchSource:0}: Error finding container 4b45081f74fcae023b539de5a9ca459f275d94d984f05cf975e5ca1283fb738d: Status 404 returned error can't find the container with id 4b45081f74fcae023b539de5a9ca459f275d94d984f05cf975e5ca1283fb738d
Jul 11 07:42:40.185371 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:40.185343    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"]
Jul 11 07:42:40.190086 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:40.190059    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode284dcd0_1354_43df_88a8_a7b6a1e8c6de.slice/crio-d3915b50f8933d87cbeaa373357d60bab16379cf480664c1c23a467371a9fcae WatchSource:0}: Error finding container d3915b50f8933d87cbeaa373357d60bab16379cf480664c1c23a467371a9fcae: Status 404 returned error can't find the container with id d3915b50f8933d87cbeaa373357d60bab16379cf480664c1c23a467371a9fcae
Jul 11 07:42:40.328502 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:40.328466    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:40.427663 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:40.427623    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:41.045477 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:41.045442    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4nnq" event={"ID":"8cba86ef-a175-40d8-ae06-54eed94ff4a5","Type":"ContainerStarted","Data":"b2bcd933bff57227672e57d9e4a4837bfcd67aa45ca58daf7069184f8cf77068"}
Jul 11 07:42:41.046971 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:41.046940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-nmbc4" event={"ID":"6be5391d-fe71-4a51-b7e0-79f9c31df6ff","Type":"ContainerStarted","Data":"4b45081f74fcae023b539de5a9ca459f275d94d984f05cf975e5ca1283fb738d"}
Jul 11 07:42:41.048287 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:41.048261    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv" event={"ID":"e284dcd0-1354-43df-88a8-a7b6a1e8c6de","Type":"ContainerStarted","Data":"d3915b50f8933d87cbeaa373357d60bab16379cf480664c1c23a467371a9fcae"}
Jul 11 07:42:41.061873 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:41.061813    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-h4nnq" podStartSLOduration=17.380124121 podStartE2EDuration="24.061791082s" totalImagesPullingTime="6.681666961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:42:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:41.060862963 +0000 UTC m=+267.264281819" watchObservedRunningTime="2026-07-11 07:42:41.061791082 +0000 UTC m=+267.265209947"
Jul 11 07:42:42.058135 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.058096    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv" event={"ID":"e284dcd0-1354-43df-88a8-a7b6a1e8c6de","Type":"ContainerStarted","Data":"00934a0e00db9d9e98e7b39689ad94c685ed2c9f58f426d9c0cde2b13e9eaba2"}
Jul 11 07:42:42.058610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.058404    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:42.063874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.063850    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv"
Jul 11 07:42:42.071787 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.071731    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-2dkrv" podStartSLOduration=3.038614969 podStartE2EDuration="4.07171511s" totalImagesPullingTime="1.033100141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:42.070590492 +0000 UTC m=+268.274009355" watchObservedRunningTime="2026-07-11 07:42:42.07171511 +0000 UTC m=+268.275133975"
Jul 11 07:42:42.713025 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.712982    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:42.754892 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.754856    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ld2gf"]
Jul 11 07:42:42.758461 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.758431    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:42.760724 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.760701    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 11 07:42:42.762034 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.761918    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 11 07:42:42.762034 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.761944    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 11 07:42:42.762034 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.761981    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-j9f8k\""
Jul 11 07:42:42.764870 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.764846    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ld2gf"]
Jul 11 07:42:42.777692 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.777658    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:42.909916 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.909876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09d813e4-f9c0-4180-9b5d-1d0089a78c34-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:42.910097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.909924    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:42.910097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.909955    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:42.910097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:42.909996    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47trt\" (UniqueName: \"kubernetes.io/projected/09d813e4-f9c0-4180-9b5d-1d0089a78c34-kube-api-access-47trt\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.011309 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.011225    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09d813e4-f9c0-4180-9b5d-1d0089a78c34-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.011309 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.011279    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.011521 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:43.011383    2572 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 11 07:42:43.011521 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.011397    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.011521 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:43.011444    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-tls podName:09d813e4-f9c0-4180-9b5d-1d0089a78c34 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:43.511423699 +0000 UTC m=+269.714842545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-tls") pod "prometheus-operator-64785b6675-ld2gf" (UID: "09d813e4-f9c0-4180-9b5d-1d0089a78c34") : secret "prometheus-operator-tls" not found
Jul 11 07:42:43.011521 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.011472    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47trt\" (UniqueName: \"kubernetes.io/projected/09d813e4-f9c0-4180-9b5d-1d0089a78c34-kube-api-access-47trt\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.012109 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.012081    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09d813e4-f9c0-4180-9b5d-1d0089a78c34-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.014168 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.014142    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.019367 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.019343    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47trt\" (UniqueName: \"kubernetes.io/projected/09d813e4-f9c0-4180-9b5d-1d0089a78c34-kube-api-access-47trt\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.137147 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.137108    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:43.515392 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.515349    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.517995 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.517971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09d813e4-f9c0-4180-9b5d-1d0089a78c34-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ld2gf\" (UID: \"09d813e4-f9c0-4180-9b5d-1d0089a78c34\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.669671 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.669635    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf"
Jul 11 07:42:43.746015 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.745926    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:43.816501 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.816471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ld2gf"]
Jul 11 07:42:43.819594 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:43.819488    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:43.820256 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:43.820235    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d813e4_f9c0_4180_9b5d_1d0089a78c34.slice/crio-262f24e0dc524f5283c978b6bcc31152edcd47b25051a47613f99fc4b7fd7699 WatchSource:0}: Error finding container 262f24e0dc524f5283c978b6bcc31152edcd47b25051a47613f99fc4b7fd7699: Status 404 returned error can't find the container with id 262f24e0dc524f5283c978b6bcc31152edcd47b25051a47613f99fc4b7fd7699
Jul 11 07:42:44.066345 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:44.066307    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf" event={"ID":"09d813e4-f9c0-4180-9b5d-1d0089a78c34","Type":"ContainerStarted","Data":"262f24e0dc524f5283c978b6bcc31152edcd47b25051a47613f99fc4b7fd7699"}
Jul 11 07:42:44.673733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:44.673699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:44.729638 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:44.729585    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:45.514208 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:45.514122    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:46.075997 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:46.075962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf" event={"ID":"09d813e4-f9c0-4180-9b5d-1d0089a78c34","Type":"ContainerStarted","Data":"ba4c760dab9766efe2a82510538a689dedfd1dedb546548ad441fec922a1e70a"}
Jul 11 07:42:46.076438 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:46.076022    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf" event={"ID":"09d813e4-f9c0-4180-9b5d-1d0089a78c34","Type":"ContainerStarted","Data":"8a8de172bc7e0f8200b70f4be002be59905079272ab0be5457138650c3e73e2b"}
Jul 11 07:42:46.091563 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:46.091494    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-ld2gf" podStartSLOduration=2.676702575 podStartE2EDuration="4.091474286s" totalImagesPullingTime="1.414771711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:46.090651414 +0000 UTC m=+272.294070278" watchObservedRunningTime="2026-07-11 07:42:46.091474286 +0000 UTC m=+272.294893152"
Jul 11 07:42:46.371105 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:46.371013    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:47.830113 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:47.830077    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.055438 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.055374    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.100882 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.100638    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cqp6t"]
Jul 11 07:42:48.104755 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.104443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.106958 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.106938    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 11 07:42:48.107518 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.107172    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 11 07:42:48.107518 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.107393    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-876tw\""
Jul 11 07:42:48.107518 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.107509    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 11 07:42:48.143089 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.142898    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.202273 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.202158    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.251958 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.251894    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-root\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.251958 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.251941    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-sys\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252027    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-wtmp\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252055    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-textfile\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252097    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edebb51e-3a83-4eb7-b990-a6e2c90786f1-metrics-client-ca\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252413 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klvw5\" (UniqueName: \"kubernetes.io/projected/edebb51e-3a83-4eb7-b990-a6e2c90786f1-kube-api-access-klvw5\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.252413 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.252217    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-tls\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.319430 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.319318    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.353433 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353360    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-sys\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353433 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353400    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-wtmp\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353433 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-textfile\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353469    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edebb51e-3a83-4eb7-b990-a6e2c90786f1-metrics-client-ca\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353499    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353584    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klvw5\" (UniqueName: \"kubernetes.io/projected/edebb51e-3a83-4eb7-b990-a6e2c90786f1-kube-api-access-klvw5\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353609    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-wtmp\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353617    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-tls\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353678    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-root\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.353707 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.354026 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-root\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.354026 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.353466    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edebb51e-3a83-4eb7-b990-a6e2c90786f1-sys\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.354194 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.354175    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-textfile\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.354542 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.354311    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.354542 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.354454    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edebb51e-3a83-4eb7-b990-a6e2c90786f1-metrics-client-ca\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.371933 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.371902    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-tls\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.371933 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.371930    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edebb51e-3a83-4eb7-b990-a6e2c90786f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.379509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.379373    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klvw5\" (UniqueName: \"kubernetes.io/projected/edebb51e-3a83-4eb7-b990-a6e2c90786f1-kube-api-access-klvw5\") pod \"node-exporter-cqp6t\" (UID: \"edebb51e-3a83-4eb7-b990-a6e2c90786f1\") " pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.418224 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.418169    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.420189 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.420154    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cqp6t"
Jul 11 07:42:48.434244 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:48.434221    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedebb51e_3a83_4eb7_b990_a6e2c90786f1.slice/crio-1279bc0fd03e9d8a184994339b336707485011e78c0531ee6ee595ae80bb4681 WatchSource:0}: Error finding container 1279bc0fd03e9d8a184994339b336707485011e78c0531ee6ee595ae80bb4681: Status 404 returned error can't find the container with id 1279bc0fd03e9d8a184994339b336707485011e78c0531ee6ee595ae80bb4681
Jul 11 07:42:48.520087 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.520046    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.639307 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.639197    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.739361 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.739321    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.915856 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.915770    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:48.957502 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.957453    2572 patch_prober.go:28] interesting pod/image-registry-7f88d7c45d-dh675 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 11 07:42:48.957675 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:48.957553    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 11 07:42:49.058110 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.058055    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:49.091670 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.091631    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cqp6t" event={"ID":"edebb51e-3a83-4eb7-b990-a6e2c90786f1","Type":"ContainerStarted","Data":"1279bc0fd03e9d8a184994339b336707485011e78c0531ee6ee595ae80bb4681"}
Jul 11 07:42:49.141231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.141196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:49.151474 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.151439    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 11 07:42:49.159942 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.159911    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.162411 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162381    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 11 07:42:49.162725 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162704    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 11 07:42:49.162808 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162707    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 11 07:42:49.163097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162935    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 11 07:42:49.163097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162953    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c7xgq\""
Jul 11 07:42:49.163097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162965    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 11 07:42:49.163097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162985    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 11 07:42:49.163097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.162938    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 11 07:42:49.163380 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.163276    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 11 07:42:49.163477 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.163457    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 11 07:42:49.172500 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.172473    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 11 07:42:49.204584 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.204549    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:49.263021 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.262986    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263120    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263165    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5kmh\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-kube-api-access-g5kmh\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263203    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263288    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263320    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263368    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263393    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263421    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263460    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-config-out\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263782 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263481    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-web-config\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.263782 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.263583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364504 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364465    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364543    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364577    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364632    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5kmh\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-kube-api-access-g5kmh\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364708    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364743    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364801    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364825    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364850    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364890    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-config-out\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.364973 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.364911    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-web-config\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.367336 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.366439    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.367336 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.366761    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.368919 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.368321    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.368919 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:49.368457    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle podName:f65bb224-f52f-46a0-b30c-039e2b16809e nodeName:}" failed. No retries permitted until 2026-07-11 07:42:49.86843782 +0000 UTC m=+276.071856667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e") : configmap references non-existent config key: ca-bundle.crt
Jul 11 07:42:49.379750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.377194    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-web-config\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.379750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.378077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.379750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.378301    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5kmh\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-kube-api-access-g5kmh\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.379750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.378941    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.379750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.379000    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.379750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.379398    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.380233 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.379867    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-config-out\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.380295 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.380282    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.381099 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.381068    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.870722 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.870678    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:49.871718 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:49.871685    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:50.074363 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:50.074320    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:42:50.982898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:50.982863    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:51.050087 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.050044    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:51.055048 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.055010    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5dbb775d55-l7slr"]
Jul 11 07:42:51.058386 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.058366    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.060723 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.060699    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 11 07:42:51.060889 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.060856    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7k79fibflmads\""
Jul 11 07:42:51.061012 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.060998    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 11 07:42:51.061079 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.061035    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 11 07:42:51.061134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.061118    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 11 07:42:51.061231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.061210    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 11 07:42:51.061231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.061231    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-b48l2\""
Jul 11 07:42:51.067680 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.067643    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dbb775d55-l7slr"]
Jul 11 07:42:51.183103 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-tls\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183118    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-grpc-tls\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183211    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afa95054-ea6f-4f5f-86a0-011942972008-metrics-client-ca\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183243    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183345    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183404    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggvt\" (UniqueName: \"kubernetes.io/projected/afa95054-ea6f-4f5f-86a0-011942972008-kube-api-access-6ggvt\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.183536 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.183435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.196701 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.196662    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:51.255042 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.254968    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:51.284232 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284195    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284410 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggvt\" (UniqueName: \"kubernetes.io/projected/afa95054-ea6f-4f5f-86a0-011942972008-kube-api-access-6ggvt\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284410 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284410 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284339    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-tls\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284410 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-grpc-tls\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284410 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284681 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284451    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afa95054-ea6f-4f5f-86a0-011942972008-metrics-client-ca\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.284681 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.284481    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.285496 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.285441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afa95054-ea6f-4f5f-86a0-011942972008-metrics-client-ca\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.287433 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.287409    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.287562 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.287459    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-tls\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.287736 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.287707    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.287838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.287818    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.288183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.288161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.288271 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.288190    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afa95054-ea6f-4f5f-86a0-011942972008-secret-grpc-tls\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.291685 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.291665    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggvt\" (UniqueName: \"kubernetes.io/projected/afa95054-ea6f-4f5f-86a0-011942972008-kube-api-access-6ggvt\") pod \"thanos-querier-5dbb775d55-l7slr\" (UID: \"afa95054-ea6f-4f5f-86a0-011942972008\") " pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.372717 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.372679    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:42:51.927696 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:51.927664    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:52.027511 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.027477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:52.120881 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.120854    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-569856469f-z6clx"]
Jul 11 07:42:52.123292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.123273    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.125927 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.125909    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 11 07:42:52.126811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.126707    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gqvhv\""
Jul 11 07:42:52.126811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.126740    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 11 07:42:52.126811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.126740    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 11 07:42:52.127015 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.126740    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 11 07:42:52.127109 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.127091    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 11 07:42:52.132107 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.132074    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 11 07:42:52.133643 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.133579    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569856469f-z6clx"]
Jul 11 07:42:52.293914 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.293838    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-oauth-config\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.293914 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.293867    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-console-config\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.294347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.293919    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcd9j\" (UniqueName: \"kubernetes.io/projected/e529ac23-052b-4904-9de7-fcc55197d411-kube-api-access-bcd9j\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.294347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.293940    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-trusted-ca-bundle\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.294347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.293955    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-oauth-serving-cert\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.294347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.294042    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-service-ca\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.294347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.294117    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-serving-cert\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395035 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.394999    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-service-ca\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395216 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.395070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-serving-cert\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395216 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.395119    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-oauth-config\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395216 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.395142    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-console-config\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395216 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.395177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcd9j\" (UniqueName: \"kubernetes.io/projected/e529ac23-052b-4904-9de7-fcc55197d411-kube-api-access-bcd9j\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395216 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.395206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-trusted-ca-bundle\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.395515 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.395228    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-oauth-serving-cert\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.396112 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.396082    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-oauth-serving-cert\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.396327 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.396276    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-console-config\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.396495 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.396449    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-trusted-ca-bundle\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.396760 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.396725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-service-ca\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.398715 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.398690    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-oauth-config\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.398907 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.398884    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-serving-cert\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.404726 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.404683    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcd9j\" (UniqueName: \"kubernetes.io/projected/e529ac23-052b-4904-9de7-fcc55197d411-kube-api-access-bcd9j\") pod \"console-569856469f-z6clx\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") " pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.434463 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.434427    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:42:52.871686 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.871613    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:52.887344 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.887171    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"]
Jul 11 07:42:52.891787 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.891763    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:52.893937 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.893910    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 11 07:42:52.894356 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.894338    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mqrqj\""
Jul 11 07:42:52.897517 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.897498    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"]
Jul 11 07:42:52.926546 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:52.926489    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:53.004957 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.004924    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cac52812-189d-4b58-ad69-2ed05defb2fc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.005132 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.004995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58m8v\" (UniqueName: \"kubernetes.io/projected/cac52812-189d-4b58-ad69-2ed05defb2fc-kube-api-access-58m8v\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.105929 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.105901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cac52812-189d-4b58-ad69-2ed05defb2fc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.106109 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.105976    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58m8v\" (UniqueName: \"kubernetes.io/projected/cac52812-189d-4b58-ad69-2ed05defb2fc-kube-api-access-58m8v\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.106109 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:53.106058    2572 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 11 07:42:53.106218 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:53.106136    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cac52812-189d-4b58-ad69-2ed05defb2fc-monitoring-plugin-cert podName:cac52812-189d-4b58-ad69-2ed05defb2fc nodeName:}" failed. No retries permitted until 2026-07-11 07:42:53.606115052 +0000 UTC m=+279.809533904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/cac52812-189d-4b58-ad69-2ed05defb2fc-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-cfb6h" (UID: "cac52812-189d-4b58-ad69-2ed05defb2fc") : secret "monitoring-plugin-cert" not found
Jul 11 07:42:53.116777 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.116746    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58m8v\" (UniqueName: \"kubernetes.io/projected/cac52812-189d-4b58-ad69-2ed05defb2fc-kube-api-access-58m8v\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.205742 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.204135    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:53.610945 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.610908    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cac52812-189d-4b58-ad69-2ed05defb2fc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.613839 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.613813    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cac52812-189d-4b58-ad69-2ed05defb2fc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-cfb6h\" (UID: \"cac52812-189d-4b58-ad69-2ed05defb2fc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:53.813401 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:53.813362    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:42:54.234488 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.234451    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:54.242557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.242246    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:42:54.245282 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.245252    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.249240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.248141    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 11 07:42:54.249240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.248414    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 11 07:42:54.249240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.248606    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 11 07:42:54.249240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.248907    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 11 07:42:54.249798 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.249669    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 11 07:42:54.249894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.249885    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 11 07:42:54.250003 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.249985    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 11 07:42:54.250901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.250101    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2gx7k\""
Jul 11 07:42:54.250901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.250160    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 11 07:42:54.250901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.250204    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 11 07:42:54.250901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.250104    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 11 07:42:54.250901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.250655    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 11 07:42:54.250901 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.250703    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 11 07:42:54.251516 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.251498    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cam22pjlgqvjo\""
Jul 11 07:42:54.251985 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.251967    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 11 07:42:54.253093 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.253075    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 11 07:42:54.259224 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.259144    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 11 07:42:54.265324 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.265224    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:42:54.317137 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317103    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317147    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317214    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317256    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317282    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317567 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317395    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-web-config\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317567 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317434    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317567 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317459    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317567 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317487    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317567 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317567 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317558    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317586    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-config\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317644    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sdsh\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-kube-api-access-9sdsh\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317675    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317725    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317749    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317796    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-config-out\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.317874 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.317831    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419542 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419487    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-web-config\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419542 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419545    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419763 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419674    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419763 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419725    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419763 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419755    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419873 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419783    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419873 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419815    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419873 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419849    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-config\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419992 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419880    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sdsh\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-kube-api-access-9sdsh\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419992 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419916    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.419992 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419970    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.419993    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420016    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-config-out\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420068    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420130    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420418 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420169    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420418 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420207    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.420418 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.420235    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.421123 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.421093    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.421695 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.421663    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.421791 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.421717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.422682    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.423049    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-web-config\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.423443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.423645    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.423942    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.424221    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.423854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.424589    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.425356    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.425657    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-config\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.425704    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.426804    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.427445    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-config-out\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.427591    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.428269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.428255    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.429152 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.429136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.430935 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.430895    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.432090 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.432070    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sdsh\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-kube-api-access-9sdsh\") pod \"prometheus-k8s-0\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.562945 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.562904    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:42:54.731246 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:54.731211    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:54.746819 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:42:54.746781    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fbmfs" podUID="b0367759-a5db-44e9-aa69-2baec901fd6a"
Jul 11 07:42:55.120444 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:55.120407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:42:56.768886 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:56.768715    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"]
Jul 11 07:42:56.771629 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:56.771600    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcac52812_189d_4b58_ad69_2ed05defb2fc.slice/crio-9a78da95a5564328500dc5a92c069dd7061fc5e301293c634fff8fb77e4ebd7d WatchSource:0}: Error finding container 9a78da95a5564328500dc5a92c069dd7061fc5e301293c634fff8fb77e4ebd7d: Status 404 returned error can't find the container with id 9a78da95a5564328500dc5a92c069dd7061fc5e301293c634fff8fb77e4ebd7d
Jul 11 07:42:56.833339 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:56.833317    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569856469f-z6clx"]
Jul 11 07:42:56.846337 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:56.846306    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode529ac23_052b_4904_9de7_fcc55197d411.slice/crio-88a9c6afa746a8ad07934e58dd611537bdb10806e176f38ce42eca868023b4fd WatchSource:0}: Error finding container 88a9c6afa746a8ad07934e58dd611537bdb10806e176f38ce42eca868023b4fd: Status 404 returned error can't find the container with id 88a9c6afa746a8ad07934e58dd611537bdb10806e176f38ce42eca868023b4fd
Jul 11 07:42:57.002512 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.002483    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dbb775d55-l7slr"]
Jul 11 07:42:57.004634 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:57.004594    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa95054_ea6f_4f5f_86a0_011942972008.slice/crio-6d4b9eb4bf5e26a8a2e7a9e863cb425542662451554c12c04850a4e93a0df12b WatchSource:0}: Error finding container 6d4b9eb4bf5e26a8a2e7a9e863cb425542662451554c12c04850a4e93a0df12b: Status 404 returned error can't find the container with id 6d4b9eb4bf5e26a8a2e7a9e863cb425542662451554c12c04850a4e93a0df12b
Jul 11 07:42:57.012419 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.012395    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 11 07:42:57.014871 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:57.014829    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf65bb224_f52f_46a0_b30c_039e2b16809e.slice/crio-b01cf885e453783a405382ca87a26a7c0ef45e1685a4966bf3d49adc6aa72ede WatchSource:0}: Error finding container b01cf885e453783a405382ca87a26a7c0ef45e1685a4966bf3d49adc6aa72ede: Status 404 returned error can't find the container with id b01cf885e453783a405382ca87a26a7c0ef45e1685a4966bf3d49adc6aa72ede
Jul 11 07:42:57.015063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.014975    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:42:57.018503 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:42:57.018481    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda149361c_6d16_4886_95f3_9394c0d55284.slice/crio-68fc66b1e2bbdffe8247bac6373d3c6edadd3e5be7d1268002142c69dbfe685b WatchSource:0}: Error finding container 68fc66b1e2bbdffe8247bac6373d3c6edadd3e5be7d1268002142c69dbfe685b: Status 404 returned error can't find the container with id 68fc66b1e2bbdffe8247bac6373d3c6edadd3e5be7d1268002142c69dbfe685b
Jul 11 07:42:57.129884 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.129799    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"6d4b9eb4bf5e26a8a2e7a9e863cb425542662451554c12c04850a4e93a0df12b"}
Jul 11 07:42:57.131303 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.131272    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569856469f-z6clx" event={"ID":"e529ac23-052b-4904-9de7-fcc55197d411","Type":"ContainerStarted","Data":"88a9c6afa746a8ad07934e58dd611537bdb10806e176f38ce42eca868023b4fd"}
Jul 11 07:42:57.132786 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.132757    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"b01cf885e453783a405382ca87a26a7c0ef45e1685a4966bf3d49adc6aa72ede"}
Jul 11 07:42:57.134195 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.134167    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h" event={"ID":"cac52812-189d-4b58-ad69-2ed05defb2fc","Type":"ContainerStarted","Data":"9a78da95a5564328500dc5a92c069dd7061fc5e301293c634fff8fb77e4ebd7d"}
Jul 11 07:42:57.135589 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.135561    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"68fc66b1e2bbdffe8247bac6373d3c6edadd3e5be7d1268002142c69dbfe685b"}
Jul 11 07:42:57.137565 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.137520    2572 generic.go:356] "Generic (PLEG): container finished" podID="edebb51e-3a83-4eb7-b990-a6e2c90786f1" containerID="f9aee475518fa12fe6a8cc84367e589728e0f67a06e6db83443b2f79f3222d36" exitCode=0
Jul 11 07:42:57.137680 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.137624    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cqp6t" event={"ID":"edebb51e-3a83-4eb7-b990-a6e2c90786f1","Type":"ContainerDied","Data":"f9aee475518fa12fe6a8cc84367e589728e0f67a06e6db83443b2f79f3222d36"}
Jul 11 07:42:57.139311 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.139279    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-nmbc4" event={"ID":"6be5391d-fe71-4a51-b7e0-79f9c31df6ff","Type":"ContainerStarted","Data":"9ded7436b2800149b29491fcc60dd4dd4b299a36f01c3e43e4f548b0d862e7a7"}
Jul 11 07:42:57.139592 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.139567    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:57.149649 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.149626    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-nmbc4"
Jul 11 07:42:57.167336 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.167275    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-nmbc4" podStartSLOduration=2.680386913 podStartE2EDuration="19.167255656s" totalImagesPullingTime="16.486868743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:57.166861365 +0000 UTC m=+283.370280230" watchObservedRunningTime="2026-07-11 07:42:57.167255656 +0000 UTC m=+283.370674519"
Jul 11 07:42:57.500008 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.499877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:57.601921 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:57.601858    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:42:58.057148 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.056721    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:42:58.076350 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.076286    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0367759-a5db-44e9-aa69-2baec901fd6a-metrics-tls\") pod \"dns-default-fbmfs\" (UID: \"b0367759-a5db-44e9-aa69-2baec901fd6a\") " pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:42:58.123787 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.123577    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbf8m\""
Jul 11 07:42:58.135811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.132267    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:42:58.155817 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.155568    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cqp6t" event={"ID":"edebb51e-3a83-4eb7-b990-a6e2c90786f1","Type":"ContainerStarted","Data":"226933d776456d4eb5e3ef3ea9e0f92737dd41cdb6d42feef3b7f183f985e34e"}
Jul 11 07:42:58.155817 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.155623    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cqp6t" event={"ID":"edebb51e-3a83-4eb7-b990-a6e2c90786f1","Type":"ContainerStarted","Data":"30ef5eaa34e6a497a0898f3f19cbe16c4bd6ee28628cda9d1c9b74bad178065f"}
Jul 11 07:42:58.173383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.172427    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cqp6t" podStartSLOduration=1.941448878 podStartE2EDuration="10.172406024s" totalImagesPullingTime="8.146243141s" totalInitContainerRuntime="84.714005ms" isStatefulPod=true podCreationTimestamp="2026-07-11 07:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:58.171962573 +0000 UTC m=+284.375381437" watchObservedRunningTime="2026-07-11 07:42:58.172406024 +0000 UTC m=+284.375824888"
Jul 11 07:42:58.973855 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.972991    2572 patch_prober.go:28] interesting pod/image-registry-7f88d7c45d-dh675 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 11 07:42:58.973855 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:42:58.973075    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 11 07:43:00.097557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:00.097503    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:00.147730 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:00.147670    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:01.729347 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:01.729305    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:01.777030 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:01.776994    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:02.149046 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:02.149013    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fbmfs"]
Jul 11 07:43:02.166937 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:43:02.166900    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0367759_a5db_44e9_aa69_2baec901fd6a.slice/crio-16556e1d94c6c408cbb2c0eda5ce172c7363571c8794143630a8f8d15c9cf810 WatchSource:0}: Error finding container 16556e1d94c6c408cbb2c0eda5ce172c7363571c8794143630a8f8d15c9cf810: Status 404 returned error can't find the container with id 16556e1d94c6c408cbb2c0eda5ce172c7363571c8794143630a8f8d15c9cf810
Jul 11 07:43:02.174722 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:02.174692    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fbmfs" event={"ID":"b0367759-a5db-44e9-aa69-2baec901fd6a","Type":"ContainerStarted","Data":"16556e1d94c6c408cbb2c0eda5ce172c7363571c8794143630a8f8d15c9cf810"}
Jul 11 07:43:03.186897 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.186842    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"5dc0a2ab6b5271310b3ea2c37012e339f9e53ac37588dc18a48900daacf5a3e0"}
Jul 11 07:43:03.187771 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.187740    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"7d9d0fd85fa0f384cc8cd822743a6369b90dd8f6b104fa5e7841f8cdc03b685d"}
Jul 11 07:43:03.187879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.187793    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"520b43466d1a93816dd16005e0430e5e630e294ee8243901b43aa9284128331f"}
Jul 11 07:43:03.188909 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.188859    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569856469f-z6clx" event={"ID":"e529ac23-052b-4904-9de7-fcc55197d411","Type":"ContainerStarted","Data":"b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f"}
Jul 11 07:43:03.191192 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.191169    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="c27deb73f64068f76463a0c40880b343fa9d662f21b7ba540f9bddac73a829ab" exitCode=0
Jul 11 07:43:03.191559 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.191452    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"c27deb73f64068f76463a0c40880b343fa9d662f21b7ba540f9bddac73a829ab"}
Jul 11 07:43:03.195401 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.194822    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h" event={"ID":"cac52812-189d-4b58-ad69-2ed05defb2fc","Type":"ContainerStarted","Data":"d5bf3304366908d509ed245e6ad0e9e42fdf9b8e288de16757af6f67546345b1"}
Jul 11 07:43:03.195401 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.195206    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:43:03.199420 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.199392    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd" exitCode=0
Jul 11 07:43:03.199738 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.199713    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd"}
Jul 11 07:43:03.201754 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.201429    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h"
Jul 11 07:43:03.206989 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.206946    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-569856469f-z6clx" podStartSLOduration=5.807289632 podStartE2EDuration="11.206927436s" totalImagesPullingTime="5.399637804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:03.205656837 +0000 UTC m=+289.409075701" watchObservedRunningTime="2026-07-11 07:43:03.206927436 +0000 UTC m=+289.410346300"
Jul 11 07:43:03.275788 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.275719    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-cfb6h" podStartSLOduration=5.812551768 podStartE2EDuration="11.275697209s" totalImagesPullingTime="5.463145441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:03.274978179 +0000 UTC m=+289.478397045" watchObservedRunningTime="2026-07-11 07:43:03.275697209 +0000 UTC m=+289.479116072"
Jul 11 07:43:03.535492 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.535407    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:03.974856 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:03.974777    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerName="registry" containerID="cri-o://6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87" gracePeriod=30
Jul 11 07:43:04.532774 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.532745    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:04.583226 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.583164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:04.761994 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.761934    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:43:04.833460 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833406    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a77e19c-65fc-4bb1-b601-895a19b8c967-ca-trust-extracted\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833460 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833455    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-certificates\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833641 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833517    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-installation-pull-secrets\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833641 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833566    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tdbk\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-kube-api-access-4tdbk\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833641 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833598    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833641 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833614    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-image-registry-private-configuration\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833849 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833653    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-bound-sa-token\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.833849 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.833670    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-trusted-ca\") pod \"3a77e19c-65fc-4bb1-b601-895a19b8c967\" (UID: \"3a77e19c-65fc-4bb1-b601-895a19b8c967\") "
Jul 11 07:43:04.834398 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.834306    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:43:04.836599 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.835266    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:43:04.838150 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.838124    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:43:04.838264 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.838241    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-kube-api-access-4tdbk" (OuterVolumeSpecName: "kube-api-access-4tdbk") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "kube-api-access-4tdbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:43:04.838826 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.838795    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:43:04.839918 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.839805    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:43:04.841342 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.841317    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:43:04.846975 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.845631    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a77e19c-65fc-4bb1-b601-895a19b8c967-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3a77e19c-65fc-4bb1-b601-895a19b8c967" (UID: "3a77e19c-65fc-4bb1-b601-895a19b8c967"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 07:43:04.935289 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935255    2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a77e19c-65fc-4bb1-b601-895a19b8c967-ca-trust-extracted\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935289 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935289    2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-certificates\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935304    2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-installation-pull-secrets\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935317    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tdbk\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-kube-api-access-4tdbk\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935330    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-registry-tls\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935346    2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a77e19c-65fc-4bb1-b601-895a19b8c967-image-registry-private-configuration\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935360    2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a77e19c-65fc-4bb1-b601-895a19b8c967-bound-sa-token\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:04.935509 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:04.935374    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a77e19c-65fc-4bb1-b601-895a19b8c967-trusted-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:05.211799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.211759    2572 generic.go:356] "Generic (PLEG): container finished" podID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerID="6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87" exitCode=0
Jul 11 07:43:05.212044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.211822    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675"
Jul 11 07:43:05.212044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.211918    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" event={"ID":"3a77e19c-65fc-4bb1-b601-895a19b8c967","Type":"ContainerDied","Data":"6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87"}
Jul 11 07:43:05.212044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.211968    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f88d7c45d-dh675" event={"ID":"3a77e19c-65fc-4bb1-b601-895a19b8c967","Type":"ContainerDied","Data":"794eb4aede19cb074e38729cdfb675a768716b71c3cb21848c54a7a56b9c9181"}
Jul 11 07:43:05.212044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.211998    2572 scope.go:117] "RemoveContainer" containerID="6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87"
Jul 11 07:43:05.217128 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.217036    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"21b2a5b87482fe6f4aabf351529a9fdc55a25e1087323a9ee39588da25719c5a"}
Jul 11 07:43:05.237581 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.237506    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f88d7c45d-dh675"]
Jul 11 07:43:05.240654 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.240623    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f88d7c45d-dh675"]
Jul 11 07:43:05.334401 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.334381    2572 scope.go:117] "RemoveContainer" containerID="6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87"
Jul 11 07:43:05.334761 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:43:05.334737    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87\": container with ID starting with 6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87 not found: ID does not exist" containerID="6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87"
Jul 11 07:43:05.334854 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:05.334777    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87"} err="failed to get container status \"6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87\": rpc error: code = NotFound desc = could not find container \"6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87\": container with ID starting with 6b6a16e9355190f19cfdedb23727d6028af6cf1be5427a3555194a7855057a87 not found: ID does not exist"
Jul 11 07:43:06.225810 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.225737    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"3cc274b9724f2c6989e281e993ba5dae047a63a87e1aee849d328c6dab3d996b"}
Jul 11 07:43:06.226169 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.225853    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" event={"ID":"afa95054-ea6f-4f5f-86a0-011942972008","Type":"ContainerStarted","Data":"af6108e9f366adab8d0204110897ccf2eaad69da5853888bb29af0ff1926f1e6"}
Jul 11 07:43:06.226169 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.225901    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:43:06.228748 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.228710    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"9e9172f98730e3efa483730f613d1efcea533581f8719d3b0da8ec365a0bf5a3"}
Jul 11 07:43:06.228888 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.228781    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"8bac06a4f34bb8859b9fff633db28156b1a1b18ad030b8e95c4a66fd7b71b872"}
Jul 11 07:43:06.228888 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.228823    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"1160f723e90c9c0cd535f615289cfc2d1a2ee66d830fffd6a4017fa7ef97852a"}
Jul 11 07:43:06.228888 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.228861    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"c40318697a43a7fc169c8cc775d45bc1b51559f64f9989a5f09d2d6f19b0540d"}
Jul 11 07:43:06.230442 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.230421    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fbmfs" event={"ID":"b0367759-a5db-44e9-aa69-2baec901fd6a","Type":"ContainerStarted","Data":"396b59425e4c91495053055806d9df9a2472a25cfd2f8783649a8c39627b8acc"}
Jul 11 07:43:06.230572 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.230466    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fbmfs" event={"ID":"b0367759-a5db-44e9-aa69-2baec901fd6a","Type":"ContainerStarted","Data":"7a71dc2e2fe87e8a7bf3d9e38a35494330fe67d06660a909e537f0eafa11ef7c"}
Jul 11 07:43:06.230572 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.230513    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:43:06.246019 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.245439    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr" podStartSLOduration=7.888118191 podStartE2EDuration="15.245416567s" totalImagesPullingTime="7.357298376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:06.244108945 +0000 UTC m=+292.447527862" watchObservedRunningTime="2026-07-11 07:43:06.245416567 +0000 UTC m=+292.448835434"
Jul 11 07:43:06.259521 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.259471    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fbmfs" podStartSLOduration=255.721168355 podStartE2EDuration="4m18.259451081s" totalImagesPullingTime="2.538282726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:06.259113757 +0000 UTC m=+292.462532621" watchObservedRunningTime="2026-07-11 07:43:06.259451081 +0000 UTC m=+292.462869944"
Jul 11 07:43:06.289928 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:43:06.289342    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jnj7r" podUID="29f420da-3869-4a78-97a2-c4e78477cfb2"
Jul 11 07:43:06.296052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:06.296027    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" path="/var/lib/kubelet/pods/3a77e19c-65fc-4bb1-b601-895a19b8c967/volumes"
Jul 11 07:43:07.239544 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:07.239491    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"fe7549a0457d92d40b685c469ca00cddcf5dd4c89ead4404f0eb5f7c75a166b0"}
Jul 11 07:43:07.240046 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:07.239598    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerStarted","Data":"1fcb491149c4c4f00d40e36ce69d51548fb846c12bba5c681f58eecb8009723a"}
Jul 11 07:43:07.246115 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:07.246092    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5dbb775d55-l7slr"
Jul 11 07:43:07.284756 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:07.284690    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.553837361 podStartE2EDuration="18.284666837s" totalImagesPullingTime="7.702833268s" totalInitContainerRuntime="27.996208ms" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:07.283715963 +0000 UTC m=+293.487134829" watchObservedRunningTime="2026-07-11 07:43:07.284666837 +0000 UTC m=+293.488085700"
Jul 11 07:43:08.246288 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:08.246258    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77"}
Jul 11 07:43:08.246624 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:08.246378    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d"}
Jul 11 07:43:08.246624 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:08.246422    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490"}
Jul 11 07:43:08.749472 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:08.749395    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:08.850045 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:08.850019    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:09.256001 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.255962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed"}
Jul 11 07:43:09.256478 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.256047    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a"}
Jul 11 07:43:09.256478 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.256085    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerStarted","Data":"4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802"}
Jul 11 07:43:09.281868 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.281800    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.439461476 podStartE2EDuration="15.281773388s" totalImagesPullingTime="9.830389721s" totalInitContainerRuntime="11.922191ms" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:09.280361365 +0000 UTC m=+295.483780255" watchObservedRunningTime="2026-07-11 07:43:09.281773388 +0000 UTC m=+295.485192301"
Jul 11 07:43:09.285028 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.285002    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:09.335538 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.335482    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:09.563656 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:09.563625    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:43:11.688932 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:11.688901    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:11.736997 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:11.736966    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:11.877070 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:11.877031    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:11.924592 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:11.924547    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:12.435399 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:12.435371    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:43:12.435591 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:12.435417    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:43:12.440035 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:12.440015    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:43:13.276137 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:13.276108    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:43:13.389251 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:13.389216    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:13.817587 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:13.817545    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:13.877214 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:13.877177    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:14.063836 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:14.063798    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:14.144596 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:14.144421    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:14.179996 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:14.179960    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:43:14.179996 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:14.179972    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:43:14.188069 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:14.188050    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 11 07:43:16.242731 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:16.242703    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fbmfs"
Jul 11 07:43:16.347805 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:16.347763    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:16.395267 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:16.395232    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:18.288262 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:18.288237    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:43:18.289689 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:18.289654    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:18.390750 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:18.390715    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:19.359576 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:19.359538    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-569856469f-z6clx"]
Jul 11 07:43:19.470576 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:19.470546    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:19.571336 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:19.571305    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:19.672145 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:19.672064    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:22.848826 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:22.848795    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:22.901553 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:22.901506    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:23.741094 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:23.741060    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:23.796415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:23.796384    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:25.529460 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:25.529428    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:25.630656 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:25.630618    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:36.865549 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:36.865493    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:36.966506 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:36.966471    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:38.740686 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:38.740653    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:38.843441 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:38.843406    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:41.383044 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:41.383013    2572 generic.go:356] "Generic (PLEG): container finished" podID="b996739e-74c6-4e14-a7f6-20594f36940d" containerID="0cf2dc797a1729518ab63debd2bae12352bddda1739f9d188368d8d4e15318fa" exitCode=0
Jul 11 07:43:41.383407 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:41.383057    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b" event={"ID":"b996739e-74c6-4e14-a7f6-20594f36940d","Type":"ContainerDied","Data":"0cf2dc797a1729518ab63debd2bae12352bddda1739f9d188368d8d4e15318fa"}
Jul 11 07:43:41.383407 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:41.383345    2572 scope.go:117] "RemoveContainer" containerID="0cf2dc797a1729518ab63debd2bae12352bddda1739f9d188368d8d4e15318fa"
Jul 11 07:43:42.389312 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:42.389279    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lzt6b" event={"ID":"b996739e-74c6-4e14-a7f6-20594f36940d","Type":"ContainerStarted","Data":"df3403d6fd8125451ae7979039884f93a2fff5a0febe5b110ffc3340ea785e08"}
Jul 11 07:43:44.379204 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.379165    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-569856469f-z6clx" podUID="e529ac23-052b-4904-9de7-fcc55197d411" containerName="console" containerID="cri-o://b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f" gracePeriod=15
Jul 11 07:43:44.654407 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.654336    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:43:44.781273 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781244    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-trusted-ca-bundle\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781303    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcd9j\" (UniqueName: \"kubernetes.io/projected/e529ac23-052b-4904-9de7-fcc55197d411-kube-api-access-bcd9j\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781336    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-oauth-config\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781351    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-service-ca\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781376    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-serving-cert\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781392    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-oauth-serving-cert\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781758 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781515    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-console-config\") pod \"e529ac23-052b-4904-9de7-fcc55197d411\" (UID: \"e529ac23-052b-4904-9de7-fcc55197d411\") "
Jul 11 07:43:44.781810 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781750    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:43:44.781875 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781833    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:43:44.781875 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781862    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-trusted-ca-bundle\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:44.781966 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.781892    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-console-config" (OuterVolumeSpecName: "console-config") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:43:44.782236 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.782212    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-service-ca" (OuterVolumeSpecName: "service-ca") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:43:44.783644 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.783617    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:43:44.783753 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.783685    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:43:44.783753 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.783719    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e529ac23-052b-4904-9de7-fcc55197d411-kube-api-access-bcd9j" (OuterVolumeSpecName: "kube-api-access-bcd9j") pod "e529ac23-052b-4904-9de7-fcc55197d411" (UID: "e529ac23-052b-4904-9de7-fcc55197d411"). InnerVolumeSpecName "kube-api-access-bcd9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:43:44.882841 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.882811    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-serving-cert\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:44.882841 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.882836    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-oauth-serving-cert\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:44.882841 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.882845    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-console-config\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:44.883047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.882854    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcd9j\" (UniqueName: \"kubernetes.io/projected/e529ac23-052b-4904-9de7-fcc55197d411-kube-api-access-bcd9j\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:44.883047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.882863    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e529ac23-052b-4904-9de7-fcc55197d411-console-oauth-config\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:44.883047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:44.882871    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e529ac23-052b-4904-9de7-fcc55197d411-service-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:43:45.402429 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.402400    2572 generic.go:356] "Generic (PLEG): container finished" podID="e529ac23-052b-4904-9de7-fcc55197d411" containerID="b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f" exitCode=0
Jul 11 07:43:45.402799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.402457    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569856469f-z6clx" event={"ID":"e529ac23-052b-4904-9de7-fcc55197d411","Type":"ContainerDied","Data":"b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f"}
Jul 11 07:43:45.402799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.402459    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569856469f-z6clx"
Jul 11 07:43:45.402799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.402494    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569856469f-z6clx" event={"ID":"e529ac23-052b-4904-9de7-fcc55197d411","Type":"ContainerDied","Data":"88a9c6afa746a8ad07934e58dd611537bdb10806e176f38ce42eca868023b4fd"}
Jul 11 07:43:45.402799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.402515    2572 scope.go:117] "RemoveContainer" containerID="b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f"
Jul 11 07:43:45.411231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.411216    2572 scope.go:117] "RemoveContainer" containerID="b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f"
Jul 11 07:43:45.411459 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:43:45.411441    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f\": container with ID starting with b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f not found: ID does not exist" containerID="b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f"
Jul 11 07:43:45.411493 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.411467    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f"} err="failed to get container status \"b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f\": rpc error: code = NotFound desc = could not find container \"b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f\": container with ID starting with b49ca11d258e6cb421874956a3b73a2456cc954399fa2e52707cc2fa9072d71f not found: ID does not exist"
Jul 11 07:43:45.421947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.421923    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-569856469f-z6clx"]
Jul 11 07:43:45.426569 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:45.426549    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-569856469f-z6clx"]
Jul 11 07:43:46.293835 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:46.293801    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e529ac23-052b-4904-9de7-fcc55197d411" path="/var/lib/kubelet/pods/e529ac23-052b-4904-9de7-fcc55197d411/volumes"
Jul 11 07:43:47.503681 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:47.503653    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:47.602172 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:47.602137    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:52.508178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:52.508145    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:52.556392 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:52.556346    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:43:54.563580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:54.563519    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:43:54.582475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:54.582451    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:43:55.458675 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:43:55.458652    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:04.110610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:04.110571    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:04.158703 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:04.158664    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:08.330323 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.330233    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 11 07:44:08.330767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.330563    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="alertmanager" containerID="cri-o://c40318697a43a7fc169c8cc775d45bc1b51559f64f9989a5f09d2d6f19b0540d" gracePeriod=120
Jul 11 07:44:08.330767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.330599    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-metric" containerID="cri-o://1fcb491149c4c4f00d40e36ce69d51548fb846c12bba5c681f58eecb8009723a" gracePeriod=120
Jul 11 07:44:08.330767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.330655    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy" containerID="cri-o://9e9172f98730e3efa483730f613d1efcea533581f8719d3b0da8ec365a0bf5a3" gracePeriod=120
Jul 11 07:44:08.330767 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.330706    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="prom-label-proxy" containerID="cri-o://fe7549a0457d92d40b685c469ca00cddcf5dd4c89ead4404f0eb5f7c75a166b0" gracePeriod=120
Jul 11 07:44:08.330992 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.330782    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-web" containerID="cri-o://8bac06a4f34bb8859b9fff633db28156b1a1b18ad030b8e95c4a66fd7b71b872" gracePeriod=120
Jul 11 07:44:08.331048 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.331007    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="config-reloader" containerID="cri-o://1160f723e90c9c0cd535f615289cfc2d1a2ee66d830fffd6a4017fa7ef97852a" gracePeriod=120
Jul 11 07:44:08.501811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501781    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="fe7549a0457d92d40b685c469ca00cddcf5dd4c89ead4404f0eb5f7c75a166b0" exitCode=0
Jul 11 07:44:08.501811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501805    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="9e9172f98730e3efa483730f613d1efcea533581f8719d3b0da8ec365a0bf5a3" exitCode=0
Jul 11 07:44:08.501811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501813    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="1160f723e90c9c0cd535f615289cfc2d1a2ee66d830fffd6a4017fa7ef97852a" exitCode=0
Jul 11 07:44:08.501811 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501818    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="c40318697a43a7fc169c8cc775d45bc1b51559f64f9989a5f09d2d6f19b0540d" exitCode=0
Jul 11 07:44:08.502052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501854    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"fe7549a0457d92d40b685c469ca00cddcf5dd4c89ead4404f0eb5f7c75a166b0"}
Jul 11 07:44:08.502052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501912    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"9e9172f98730e3efa483730f613d1efcea533581f8719d3b0da8ec365a0bf5a3"}
Jul 11 07:44:08.502052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501937    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"1160f723e90c9c0cd535f615289cfc2d1a2ee66d830fffd6a4017fa7ef97852a"}
Jul 11 07:44:08.502052 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:08.501960    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"c40318697a43a7fc169c8cc775d45bc1b51559f64f9989a5f09d2d6f19b0540d"}
Jul 11 07:44:09.510929 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.510904    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="1fcb491149c4c4f00d40e36ce69d51548fb846c12bba5c681f58eecb8009723a" exitCode=0
Jul 11 07:44:09.510929 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.510924    2572 generic.go:356] "Generic (PLEG): container finished" podID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerID="8bac06a4f34bb8859b9fff633db28156b1a1b18ad030b8e95c4a66fd7b71b872" exitCode=0
Jul 11 07:44:09.511263 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.510959    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"1fcb491149c4c4f00d40e36ce69d51548fb846c12bba5c681f58eecb8009723a"}
Jul 11 07:44:09.511263 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.511009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"8bac06a4f34bb8859b9fff633db28156b1a1b18ad030b8e95c4a66fd7b71b872"}
Jul 11 07:44:09.578207 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.578183    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:44:09.696082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.695997    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-config-volume\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696035    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696060    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-main-db\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696077    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-web-config\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696099    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-web\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696121    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-cluster-tls-config\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696160    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-main-tls\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696214    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696250    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-config-out\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696275    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-tls-assets\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696336    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5kmh\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-kube-api-access-g5kmh\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696360    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-metrics-client-ca\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696387    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle\") pod \"f65bb224-f52f-46a0-b30c-039e2b16809e\" (UID: \"f65bb224-f52f-46a0-b30c-039e2b16809e\") "
Jul 11 07:44:09.696865 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696458    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 07:44:09.696865 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696787    2572 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-main-db\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.696865 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.696859    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:09.698904 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.698874    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-config-volume" (OuterVolumeSpecName: "config-volume") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.699085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.699057    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.699196 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.699155    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.699422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.699371    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:09.699422 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.699387    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:44:09.699616 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.699594    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.699756 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.699737    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-kube-api-access-g5kmh" (OuterVolumeSpecName: "kube-api-access-g5kmh") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "kube-api-access-g5kmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:44:09.700943 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.700923    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-config-out" (OuterVolumeSpecName: "config-out") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 07:44:09.701122 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.701101    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.703082 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.703063    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.708814 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.708790    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-web-config" (OuterVolumeSpecName: "web-config") pod "f65bb224-f52f-46a0-b30c-039e2b16809e" (UID: "f65bb224-f52f-46a0-b30c-039e2b16809e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:09.797387 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797353    2572 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-config-volume\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797387 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797384    2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797398    2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-web-config\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797411    2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797422    2572 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-cluster-tls-config\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797436    2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-main-tls\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797448    2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f65bb224-f52f-46a0-b30c-039e2b16809e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797460    2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f65bb224-f52f-46a0-b30c-039e2b16809e-config-out\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797472    2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-tls-assets\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797482    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5kmh\" (UniqueName: \"kubernetes.io/projected/f65bb224-f52f-46a0-b30c-039e2b16809e-kube-api-access-g5kmh\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797493    2572 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-metrics-client-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:09.797626 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:09.797507    2572 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f65bb224-f52f-46a0-b30c-039e2b16809e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:10.517851 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.517818    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f65bb224-f52f-46a0-b30c-039e2b16809e","Type":"ContainerDied","Data":"b01cf885e453783a405382ca87a26a7c0ef45e1685a4966bf3d49adc6aa72ede"}
Jul 11 07:44:10.518273 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.517866    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 11 07:44:10.518273 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.517908    2572 scope.go:117] "RemoveContainer" containerID="fe7549a0457d92d40b685c469ca00cddcf5dd4c89ead4404f0eb5f7c75a166b0"
Jul 11 07:44:10.525503 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.525486    2572 scope.go:117] "RemoveContainer" containerID="1fcb491149c4c4f00d40e36ce69d51548fb846c12bba5c681f58eecb8009723a"
Jul 11 07:44:10.532705 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.532686    2572 scope.go:117] "RemoveContainer" containerID="9e9172f98730e3efa483730f613d1efcea533581f8719d3b0da8ec365a0bf5a3"
Jul 11 07:44:10.537351 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.537321    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 11 07:44:10.541024 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.541008    2572 scope.go:117] "RemoveContainer" containerID="8bac06a4f34bb8859b9fff633db28156b1a1b18ad030b8e95c4a66fd7b71b872"
Jul 11 07:44:10.541412 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.541391    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 11 07:44:10.549073 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.549053    2572 scope.go:117] "RemoveContainer" containerID="1160f723e90c9c0cd535f615289cfc2d1a2ee66d830fffd6a4017fa7ef97852a"
Jul 11 07:44:10.556283 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.556264    2572 scope.go:117] "RemoveContainer" containerID="c40318697a43a7fc169c8cc775d45bc1b51559f64f9989a5f09d2d6f19b0540d"
Jul 11 07:44:10.563925 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:10.563910    2572 scope.go:117] "RemoveContainer" containerID="c27deb73f64068f76463a0c40880b343fa9d662f21b7ba540f9bddac73a829ab"
Jul 11 07:44:12.295363 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.295332    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" path="/var/lib/kubelet/pods/f65bb224-f52f-46a0-b30c-039e2b16809e/volumes"
Jul 11 07:44:12.548014 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.547927    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:44:12.548424 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.548204    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="prometheus" containerID="cri-o://d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490" gracePeriod=600
Jul 11 07:44:12.548424 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.548265    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed" gracePeriod=600
Jul 11 07:44:12.548424 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.548266    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="thanos-sidecar" containerID="cri-o://b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77" gracePeriod=600
Jul 11 07:44:12.548424 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.548294    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="config-reloader" containerID="cri-o://75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d" gracePeriod=600
Jul 11 07:44:12.548424 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.548233    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy" containerID="cri-o://c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a" gracePeriod=600
Jul 11 07:44:12.548424 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:12.548271    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-web" containerID="cri-o://4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802" gracePeriod=600
Jul 11 07:44:13.533949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.533916    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed" exitCode=0
Jul 11 07:44:13.533949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.533940    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a" exitCode=0
Jul 11 07:44:13.533949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.533949    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77" exitCode=0
Jul 11 07:44:13.533949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.533954    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d" exitCode=0
Jul 11 07:44:13.533949 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.533959    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490" exitCode=0
Jul 11 07:44:13.534507 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.533994    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed"}
Jul 11 07:44:13.534507 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.534112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a"}
Jul 11 07:44:13.534507 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.534153    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77"}
Jul 11 07:44:13.534507 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.534191    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d"}
Jul 11 07:44:13.534507 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.534231    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490"}
Jul 11 07:44:13.788300 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.788249    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:13.934749 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934715    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sdsh\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-kube-api-access-9sdsh\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934760    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934779    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-0\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934809    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-1\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934837    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-tls-assets\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934863    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-tls\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934884    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-db\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.934941 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934912    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-kube-rbac-proxy\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934951    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-kubelet-serving-ca-bundle\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934977    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-grpc-tls\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.934998    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-config-out\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935035    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935065    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-metrics-client-certs\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935093    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-2\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935122    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-metrics-client-ca\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935146    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-trusted-ca-bundle\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935181    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-serving-certs-ca-bundle\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935206    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-config\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935227    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-web-config\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935275 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935249    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-thanos-prometheus-http-client-file\") pod \"a149361c-6d16-4886-95f3-9394c0d55284\" (UID: \"a149361c-6d16-4886-95f3-9394c0d55284\") "
Jul 11 07:44:13.935834 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935272    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.935834 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935550    2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:13.935834 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.935544    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.936249 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.936222    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.936599 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.936570    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.936769 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.936743    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.937060 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.937037    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.937152 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.937109    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 07:44:13.937211 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.937189    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 07:44:13.938167 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.938138    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-kube-api-access-9sdsh" (OuterVolumeSpecName: "kube-api-access-9sdsh") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "kube-api-access-9sdsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:44:13.938935 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.938899    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.939235 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.939202    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:44:13.939369 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.939288    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-config-out" (OuterVolumeSpecName: "config-out") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 07:44:13.939369 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.939353    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-config" (OuterVolumeSpecName: "config") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.939490 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.939438    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.939932 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.939903    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.940029 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.939936    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.940123 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.940105    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.940295 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.940271    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.940990 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.940973    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:13.949468 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:13.949447    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-web-config" (OuterVolumeSpecName: "web-config") pod "a149361c-6d16-4886-95f3-9394c0d55284" (UID: "a149361c-6d16-4886-95f3-9394c0d55284"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 07:44:14.036485 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036447    2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036485 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036478    2572 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-metrics-client-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036485 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036491    2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036503    2572 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-metrics-client-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036516    2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036550    2572 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036562    2572 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-config\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036573    2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-web-config\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036583    2572 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036595    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sdsh\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-kube-api-access-9sdsh\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036606    2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036617    2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036629    2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a149361c-6d16-4886-95f3-9394c0d55284-tls-assets\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036647    2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036658    2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-prometheus-k8s-db\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036670    2572 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-kube-rbac-proxy\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036682    2572 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a149361c-6d16-4886-95f3-9394c0d55284-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036697    2572 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a149361c-6d16-4886-95f3-9394c0d55284-secret-grpc-tls\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.036772 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.036710    2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a149361c-6d16-4886-95f3-9394c0d55284-config-out\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:44:14.540733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.540649    2572 generic.go:356] "Generic (PLEG): container finished" podID="a149361c-6d16-4886-95f3-9394c0d55284" containerID="4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802" exitCode=0
Jul 11 07:44:14.540733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.540700    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802"}
Jul 11 07:44:14.540733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.540724    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.541176 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.540754    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a149361c-6d16-4886-95f3-9394c0d55284","Type":"ContainerDied","Data":"68fc66b1e2bbdffe8247bac6373d3c6edadd3e5be7d1268002142c69dbfe685b"}
Jul 11 07:44:14.541176 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.540785    2572 scope.go:117] "RemoveContainer" containerID="bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed"
Jul 11 07:44:14.548171 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.548154    2572 scope.go:117] "RemoveContainer" containerID="c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a"
Jul 11 07:44:14.554711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.554695    2572 scope.go:117] "RemoveContainer" containerID="4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802"
Jul 11 07:44:14.560258 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.560236    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:44:14.561203 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.561190    2572 scope.go:117] "RemoveContainer" containerID="b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77"
Jul 11 07:44:14.564377 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.564359    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:44:14.567736 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.567720    2572 scope.go:117] "RemoveContainer" containerID="75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d"
Jul 11 07:44:14.573917 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.573899    2572 scope.go:117] "RemoveContainer" containerID="d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490"
Jul 11 07:44:14.580436 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.580420    2572 scope.go:117] "RemoveContainer" containerID="a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd"
Jul 11 07:44:14.586465 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.586447    2572 scope.go:117] "RemoveContainer" containerID="bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed"
Jul 11 07:44:14.586718 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.586700    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed\": container with ID starting with bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed not found: ID does not exist" containerID="bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed"
Jul 11 07:44:14.586758 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.586727    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed"} err="failed to get container status \"bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed\": rpc error: code = NotFound desc = could not find container \"bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed\": container with ID starting with bdc3f53a780502641e940a14e51321dda481aa6bedfc2f9d189eb4f0c92b13ed not found: ID does not exist"
Jul 11 07:44:14.586758 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.586745    2572 scope.go:117] "RemoveContainer" containerID="c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a"
Jul 11 07:44:14.587002 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.586985    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a\": container with ID starting with c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a not found: ID does not exist" containerID="c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a"
Jul 11 07:44:14.587049 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587006    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a"} err="failed to get container status \"c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a\": rpc error: code = NotFound desc = could not find container \"c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a\": container with ID starting with c4ca7ced20ea9e56075663d55dbfe22256a235171d7267d6e0323533e0e1917a not found: ID does not exist"
Jul 11 07:44:14.587049 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587021    2572 scope.go:117] "RemoveContainer" containerID="4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802"
Jul 11 07:44:14.587249 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.587230    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802\": container with ID starting with 4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802 not found: ID does not exist" containerID="4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802"
Jul 11 07:44:14.587283 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587254    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802"} err="failed to get container status \"4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802\": rpc error: code = NotFound desc = could not find container \"4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802\": container with ID starting with 4d428026bb7868acf2f0cf4713fd6f333255cba0cf711df20c34c8816b909802 not found: ID does not exist"
Jul 11 07:44:14.587283 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587265    2572 scope.go:117] "RemoveContainer" containerID="b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77"
Jul 11 07:44:14.587470 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.587450    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77\": container with ID starting with b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77 not found: ID does not exist" containerID="b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77"
Jul 11 07:44:14.587517 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587474    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77"} err="failed to get container status \"b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77\": rpc error: code = NotFound desc = could not find container \"b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77\": container with ID starting with b7e1c0dd94e35ed69f713af0bf83d44d76ec9992af8e31811c60b7048427fd77 not found: ID does not exist"
Jul 11 07:44:14.587517 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587486    2572 scope.go:117] "RemoveContainer" containerID="75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d"
Jul 11 07:44:14.587787 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.587762    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d\": container with ID starting with 75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d not found: ID does not exist" containerID="75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d"
Jul 11 07:44:14.587838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587790    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d"} err="failed to get container status \"75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d\": rpc error: code = NotFound desc = could not find container \"75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d\": container with ID starting with 75c6b2c615a30397b79ca8c80b65795578041dec382a59017bd8a1671ccebe7d not found: ID does not exist"
Jul 11 07:44:14.587838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.587802    2572 scope.go:117] "RemoveContainer" containerID="d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490"
Jul 11 07:44:14.588009 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.587995    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490\": container with ID starting with d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490 not found: ID does not exist" containerID="d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490"
Jul 11 07:44:14.588047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.588012    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490"} err="failed to get container status \"d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490\": rpc error: code = NotFound desc = could not find container \"d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490\": container with ID starting with d42f3a219d31511b0df196298299ed81be8e78e855a77d9a1f4ec346e5ae1490 not found: ID does not exist"
Jul 11 07:44:14.588047 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.588023    2572 scope.go:117] "RemoveContainer" containerID="a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd"
Jul 11 07:44:14.588216 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:44:14.588201    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd\": container with ID starting with a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd not found: ID does not exist" containerID="a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd"
Jul 11 07:44:14.588253 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.588219    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd"} err="failed to get container status \"a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd\": rpc error: code = NotFound desc = could not find container \"a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd\": container with ID starting with a8e5f6af97f312aee4e35e24383c87e01567d967c2d3b974b3487a47331624fd not found: ID does not exist"
Jul 11 07:44:14.590220 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590202    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:44:14.590469 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590457    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590471    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590482    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590488    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590498    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="prom-label-proxy"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590503    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="prom-label-proxy"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590509    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-web"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590514    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-web"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590536    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="thanos-sidecar"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590542    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="thanos-sidecar"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590549    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-thanos"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590555    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-thanos"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590563    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e529ac23-052b-4904-9de7-fcc55197d411" containerName="console"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590568    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e529ac23-052b-4904-9de7-fcc55197d411" containerName="console"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590573    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="config-reloader"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590578    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="config-reloader"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590583    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="init-config-reloader"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590588    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="init-config-reloader"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590596    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="prometheus"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590600    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="prometheus"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590606    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerName="registry"
Jul 11 07:44:14.590602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590611    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerName="registry"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590617    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-web"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590622    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-web"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590631    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="alertmanager"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590635    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="alertmanager"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590641    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-metric"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590645    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-metric"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590651    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="config-reloader"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590655    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="config-reloader"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590661    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="init-config-reloader"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590666    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="init-config-reloader"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590711    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="config-reloader"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590718    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-thanos"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590723    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590727    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="prometheus"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590733    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy-web"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590738    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="kube-rbac-proxy"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590743    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e529ac23-052b-4904-9de7-fcc55197d411" containerName="console"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590747    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="prom-label-proxy"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590752    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="alertmanager"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590757    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-web"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590763    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65bb224-f52f-46a0-b30c-039e2b16809e" containerName="kube-rbac-proxy-metric"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590768    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="thanos-sidecar"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590774    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a149361c-6d16-4886-95f3-9394c0d55284" containerName="config-reloader"
Jul 11 07:44:14.591175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.590779    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a77e19c-65fc-4bb1-b601-895a19b8c967" containerName="registry"
Jul 11 07:44:14.596503 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.596470    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.599415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599396    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 11 07:44:14.599519 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599449    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2gx7k\""
Jul 11 07:44:14.599519 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599402    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 11 07:44:14.599519 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599402    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 11 07:44:14.600151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599915    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 11 07:44:14.600151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599951    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 11 07:44:14.600151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599984    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 11 07:44:14.600151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.599967    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 11 07:44:14.600151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.600007    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 11 07:44:14.600151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.600020    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 11 07:44:14.600491 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.600475    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cam22pjlgqvjo\""
Jul 11 07:44:14.600651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.600627    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 11 07:44:14.600710 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.600642    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 11 07:44:14.600890 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.600871    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 11 07:44:14.601072 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.601058    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 11 07:44:14.602936 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.602913    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 11 07:44:14.606638 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.606227    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 11 07:44:14.608716 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.608695    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:44:14.742108 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8a224f6-56ca-4dea-a3f5-8cec3e932390-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742108 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742108    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742126    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742146    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742180    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742214    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8a224f6-56ca-4dea-a3f5-8cec3e932390-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742278 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742420 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742420 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742310    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742420 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742367    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742420 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742388    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-config\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742423    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742443    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742457    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742478    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742557 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742517    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742698 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742566    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742698 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742585    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742698 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742603    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.742698 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.742616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m44vs\" (UniqueName: \"kubernetes.io/projected/c8a224f6-56ca-4dea-a3f5-8cec3e932390-kube-api-access-m44vs\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843481 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843438    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843481 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843487    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-config\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843509    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843580    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843606    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843885 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843744    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.843885 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.843772    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844138 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844107    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844213 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844213 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844161    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m44vs\" (UniqueName: \"kubernetes.io/projected/c8a224f6-56ca-4dea-a3f5-8cec3e932390-kube-api-access-m44vs\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844220    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8a224f6-56ca-4dea-a3f5-8cec3e932390-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844245    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844268    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844301    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844322    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844363    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8a224f6-56ca-4dea-a3f5-8cec3e932390-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844402    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844431    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.844514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.844459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.845193 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.845170    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.846947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.846744    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-config\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.846947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.846836    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.846947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.846836    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.846947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.846837    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.847423 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.847397    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.847552 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.847504    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.848284 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.847881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.848284 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.847972    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.848284 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.848085    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.848284 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.848225    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8a224f6-56ca-4dea-a3f5-8cec3e932390-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.848284 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.848274    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.848618 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.848406    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.849073 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.849052    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8a224f6-56ca-4dea-a3f5-8cec3e932390-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.849599 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.849581    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.849997 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.849979    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8a224f6-56ca-4dea-a3f5-8cec3e932390-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.850063 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.850031    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8a224f6-56ca-4dea-a3f5-8cec3e932390-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.854563 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.854504    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m44vs\" (UniqueName: \"kubernetes.io/projected/c8a224f6-56ca-4dea-a3f5-8cec3e932390-kube-api-access-m44vs\") pod \"prometheus-k8s-0\" (UID: \"c8a224f6-56ca-4dea-a3f5-8cec3e932390\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:14.909476 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:14.909439    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:15.050292 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:15.050262    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 11 07:44:15.053835 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:44:15.053811    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a224f6_56ca_4dea_a3f5_8cec3e932390.slice/crio-de56e579e7c922715b38cc25ca94eeab48992ed5eb75477273f389bafabf1f09 WatchSource:0}: Error finding container de56e579e7c922715b38cc25ca94eeab48992ed5eb75477273f389bafabf1f09: Status 404 returned error can't find the container with id de56e579e7c922715b38cc25ca94eeab48992ed5eb75477273f389bafabf1f09
Jul 11 07:44:15.547798 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:15.547693    2572 generic.go:356] "Generic (PLEG): container finished" podID="c8a224f6-56ca-4dea-a3f5-8cec3e932390" containerID="45bab2739d212802a995154f160a68e8c15f83ff57e59382aebd6f083421dd93" exitCode=0
Jul 11 07:44:15.547798 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:15.547782    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerDied","Data":"45bab2739d212802a995154f160a68e8c15f83ff57e59382aebd6f083421dd93"}
Jul 11 07:44:15.548250 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:15.547849    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"de56e579e7c922715b38cc25ca94eeab48992ed5eb75477273f389bafabf1f09"}
Jul 11 07:44:16.011373 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.011343    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:16.056641 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.056598    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:16.293175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.293146    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a149361c-6d16-4886-95f3-9394c0d55284" path="/var/lib/kubelet/pods/a149361c-6d16-4886-95f3-9394c0d55284/volumes"
Jul 11 07:44:16.555672 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.555641    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"ca96140d63be1d7af3540f798801064a201507b93d10ff40e6b19c79e3ce1076"}
Jul 11 07:44:16.556013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.555703    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"347f91c46ce90bc4c7d72ccb3eeadbbbeec6fa0dfc3d42cb6fb9d309d84760a6"}
Jul 11 07:44:16.556013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.555727    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"fcab25110c8a8d15f02ee340cf73a4122cdbebc43778973365faf8d953dda0f7"}
Jul 11 07:44:16.556013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.555751    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"87bc5f193fc04bdecba91b08038265595e6ba906b5c7c4c3f8a6ea6952df6a94"}
Jul 11 07:44:16.556013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.555773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"0f885761955ec68875d34bd2a6e65b74d115d24d2ddcd3c56a3e886280230b13"}
Jul 11 07:44:16.556013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.555803    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8a224f6-56ca-4dea-a3f5-8cec3e932390","Type":"ContainerStarted","Data":"11790489f9925a70093a881b9758c037490bd05b2ed97897f4574ae098998d9b"}
Jul 11 07:44:16.582175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:16.582116    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.563046287 podStartE2EDuration="2.582098117s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.05183ms" isStatefulPod=false podCreationTimestamp="2026-07-11 07:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:44:16.580311997 +0000 UTC m=+362.783730914" watchObservedRunningTime="2026-07-11 07:44:16.582098117 +0000 UTC m=+362.785516979"
Jul 11 07:44:19.909886 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:19.909837    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:44:21.067488 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:21.067445    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:21.115322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:21.115291    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:21.217125 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:21.217093    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:21.262049 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:21.262016    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:21.817492 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:21.817444    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:21.918221 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:21.918192    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:23.756026 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:23.755992    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:23.853127 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:23.853096    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:23.906648 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:23.906620    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:26.789299 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:26.789268    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:26.843911 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:26.843882    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:28.152582 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.152512    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:44:28.154710 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.154690    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29f420da-3869-4a78-97a2-c4e78477cfb2-metrics-certs\") pod \"network-metrics-daemon-jnj7r\" (UID: \"29f420da-3869-4a78-97a2-c4e78477cfb2\") " pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:44:28.192345 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.192322    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs4vf\""
Jul 11 07:44:28.199993 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.199977    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnj7r"
Jul 11 07:44:28.324910 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.324771    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jnj7r"]
Jul 11 07:44:28.326812 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:44:28.326784    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f420da_3869_4a78_97a2_c4e78477cfb2.slice/crio-eced1bb1b2f75a56ed678257c13ef6654bef64984082ea91b20ad8fc5aa5fe0c WatchSource:0}: Error finding container eced1bb1b2f75a56ed678257c13ef6654bef64984082ea91b20ad8fc5aa5fe0c: Status 404 returned error can't find the container with id eced1bb1b2f75a56ed678257c13ef6654bef64984082ea91b20ad8fc5aa5fe0c
Jul 11 07:44:28.328849 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.328830    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 07:44:28.603873 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:28.603843    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnj7r" event={"ID":"29f420da-3869-4a78-97a2-c4e78477cfb2","Type":"ContainerStarted","Data":"eced1bb1b2f75a56ed678257c13ef6654bef64984082ea91b20ad8fc5aa5fe0c"}
Jul 11 07:44:29.609830 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:29.609795    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnj7r" event={"ID":"29f420da-3869-4a78-97a2-c4e78477cfb2","Type":"ContainerStarted","Data":"86b8aa179ce29763dec79241d09b0c975693dd51f0f72eea179033db3eaf5d7e"}
Jul 11 07:44:29.610243 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:29.609847    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnj7r" event={"ID":"29f420da-3869-4a78-97a2-c4e78477cfb2","Type":"ContainerStarted","Data":"8784f5b9b46aab193f2fdc74ae2c2464e05314e54672d34621a048646cf0da11"}
Jul 11 07:44:29.624646 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:29.624588    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jnj7r" podStartSLOduration=374.850460384 podStartE2EDuration="6m15.624571582s" totalImagesPullingTime="774.111198ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:44:29.622868725 +0000 UTC m=+375.826287606" watchObservedRunningTime="2026-07-11 07:44:29.624571582 +0000 UTC m=+375.827990501"
Jul 11 07:44:29.706912 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:29.706881    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:29.757201 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:29.757161    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:32.202307 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:32.202272    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:32.250195 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:32.250159    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:32.303183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:32.303155    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:44:32.405150 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:44:32.405116    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:01.009349 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:01.009315    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:01.086544 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:01.086491    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:14.910334 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:14.910298    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:45:14.925458 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:14.925434    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:45:15.797100 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:15.797065    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 11 07:45:22.685636 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:22.685555    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:27.513353 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:27.513314    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:28.318988 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:28.318946    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:31.117933 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:31.117890    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:31.218420 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:31.218383    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:32.677100 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:32.677066    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:32.784986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:32.784944    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:33.473805 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:33.473764    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:33.565791 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:33.565762    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:35.466436 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:35.466405    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:35.868862 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:35.868826    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:37.915286 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:37.915255    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:38.020355 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:38.020326    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:41.523308 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:41.523271    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:41.624209 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:41.624173    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:44.079809 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:44.079770    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:45.015843 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:45.015814    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:45.064257 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:45.064218    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:48.104682 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:48.104647    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:48.204699 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:48.204671    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:45:49.927235 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:45:49.927203    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:06.353254 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:06.353218    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:06.440915 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:06.440880    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:07.137573 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:07.137509    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:07.236146 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:07.236109    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:07.728666 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:07.728621    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:07.913687 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:07.913651    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.019687 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.019609    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.089589 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.089559    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"]
Jul 11 07:46:08.092569 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.092551    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.096178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.096149    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 11 07:46:08.096303 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.096286    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 11 07:46:08.096407 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.096389    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n26xj\""
Jul 11 07:46:08.096818 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.096804    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 11 07:46:08.097104 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.097093    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 11 07:46:08.110320 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.110294    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"]
Jul 11 07:46:08.274500 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.274414    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s6fg\" (UniqueName: \"kubernetes.io/projected/acbe2277-11bc-49cc-92e7-eb0483fbbecb-kube-api-access-8s6fg\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.274665 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.274496    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acbe2277-11bc-49cc-92e7-eb0483fbbecb-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.274665 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.274583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/acbe2277-11bc-49cc-92e7-eb0483fbbecb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.274665 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.274604    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acbe2277-11bc-49cc-92e7-eb0483fbbecb-tmp\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.274665 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.274660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/acbe2277-11bc-49cc-92e7-eb0483fbbecb-manifests\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.299076 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.299046    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.375785 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.375748    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acbe2277-11bc-49cc-92e7-eb0483fbbecb-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.375950 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.375812    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/acbe2277-11bc-49cc-92e7-eb0483fbbecb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.375950 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.375858    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acbe2277-11bc-49cc-92e7-eb0483fbbecb-tmp\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.375950 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.375904    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/acbe2277-11bc-49cc-92e7-eb0483fbbecb-manifests\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.375950 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.375931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s6fg\" (UniqueName: \"kubernetes.io/projected/acbe2277-11bc-49cc-92e7-eb0483fbbecb-kube-api-access-8s6fg\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.376348 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.376325    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/acbe2277-11bc-49cc-92e7-eb0483fbbecb-manifests\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.378118 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.378087    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acbe2277-11bc-49cc-92e7-eb0483fbbecb-tmp\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.378314 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.378293    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/acbe2277-11bc-49cc-92e7-eb0483fbbecb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.378423 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.378405    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acbe2277-11bc-49cc-92e7-eb0483fbbecb-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.383577 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.383557    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s6fg\" (UniqueName: \"kubernetes.io/projected/acbe2277-11bc-49cc-92e7-eb0483fbbecb-kube-api-access-8s6fg\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-rgq2l\" (UID: \"acbe2277-11bc-49cc-92e7-eb0483fbbecb\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.399570 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.399493    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.402411 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.402388    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:08.493269 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.493238    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.542899 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.542819    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"]
Jul 11 07:46:08.546317 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:46:08.546287    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacbe2277_11bc_49cc_92e7_eb0483fbbecb.slice/crio-921b113672805b24df47d4da1e9fedafe8baa7d266298c2f1dc8fac754a03aa8 WatchSource:0}: Error finding container 921b113672805b24df47d4da1e9fedafe8baa7d266298c2f1dc8fac754a03aa8: Status 404 returned error can't find the container with id 921b113672805b24df47d4da1e9fedafe8baa7d266298c2f1dc8fac754a03aa8
Jul 11 07:46:08.594476 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.594444    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.694385 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.694348    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.794456 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.794378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:08.982448 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:08.982411    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l" event={"ID":"acbe2277-11bc-49cc-92e7-eb0483fbbecb","Type":"ContainerStarted","Data":"921b113672805b24df47d4da1e9fedafe8baa7d266298c2f1dc8fac754a03aa8"}
Jul 11 07:46:11.998040 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:11.998007    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l" event={"ID":"acbe2277-11bc-49cc-92e7-eb0483fbbecb","Type":"ContainerStarted","Data":"e88ecb67972a81c4f50b02c95c33a39f633b96fb69c3cde61657ed0c20a5aecc"}
Jul 11 07:46:13.003205 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:13.003105    2572 generic.go:356] "Generic (PLEG): container finished" podID="acbe2277-11bc-49cc-92e7-eb0483fbbecb" containerID="e88ecb67972a81c4f50b02c95c33a39f633b96fb69c3cde61657ed0c20a5aecc" exitCode=0
Jul 11 07:46:13.003205 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:13.003177    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l" event={"ID":"acbe2277-11bc-49cc-92e7-eb0483fbbecb","Type":"ContainerDied","Data":"e88ecb67972a81c4f50b02c95c33a39f633b96fb69c3cde61657ed0c20a5aecc"}
Jul 11 07:46:14.011323 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:14.011279    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l" event={"ID":"acbe2277-11bc-49cc-92e7-eb0483fbbecb","Type":"ContainerStarted","Data":"fc03b7726ea61677296b098acffa870c6acd87a3fe4266e19e9155ee8e18a66f"}
Jul 11 07:46:14.011714 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:14.011492    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:14.030313 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:14.030261    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l" podStartSLOduration=1.726544316 podStartE2EDuration="6.030247873s" totalImagesPullingTime="3.236328702s" totalInitContainerRuntime="1.067374855s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:14.030011965 +0000 UTC m=+480.233430832" watchObservedRunningTime="2026-07-11 07:46:14.030247873 +0000 UTC m=+480.233666735"
Jul 11 07:46:20.200380 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:20.200342    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:20.299737 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:20.299702    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:25.016919 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.016892    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-rgq2l"
Jul 11 07:46:25.140350 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.140317    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:25.235630 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.235595    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"]
Jul 11 07:46:25.245748 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.245727    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.248602 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.248575    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:25.248742 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.248720    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 11 07:46:25.248921 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.248901    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 11 07:46:25.249129 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.249108    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4gpvv\""
Jul 11 07:46:25.249272 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.249253    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 11 07:46:25.249415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.249401    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 11 07:46:25.249489 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.249472    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 11 07:46:25.253487 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.253471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"]
Jul 11 07:46:25.313538 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.313493    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glbr\" (UniqueName: \"kubernetes.io/projected/9b049884-7d4d-40ca-a508-1d294895d351-kube-api-access-5glbr\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.313687 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.313554    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b049884-7d4d-40ca-a508-1d294895d351-cert\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.313687 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.313582    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b049884-7d4d-40ca-a508-1d294895d351-metrics-cert\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.313687 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.313641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b049884-7d4d-40ca-a508-1d294895d351-manager-config\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.414990 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.414956    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b049884-7d4d-40ca-a508-1d294895d351-manager-config\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.415165 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.414995    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glbr\" (UniqueName: \"kubernetes.io/projected/9b049884-7d4d-40ca-a508-1d294895d351-kube-api-access-5glbr\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.415165 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.415030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b049884-7d4d-40ca-a508-1d294895d351-cert\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.415165 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.415058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b049884-7d4d-40ca-a508-1d294895d351-metrics-cert\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.415660 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.415634    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b049884-7d4d-40ca-a508-1d294895d351-manager-config\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.417499 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.417475    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b049884-7d4d-40ca-a508-1d294895d351-cert\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.417608 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.417504    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b049884-7d4d-40ca-a508-1d294895d351-metrics-cert\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.423158 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.423136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glbr\" (UniqueName: \"kubernetes.io/projected/9b049884-7d4d-40ca-a508-1d294895d351-kube-api-access-5glbr\") pod \"lws-controller-manager-697d57cd68-2wm5n\" (UID: \"9b049884-7d4d-40ca-a508-1d294895d351\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.555491 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.555459    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:25.684258 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:25.684079    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"]
Jul 11 07:46:25.686935 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:46:25.686906    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b049884_7d4d_40ca_a508_1d294895d351.slice/crio-c4fdca3b1dca88d18adc5bee04d5b61a9756b6bac1c82a5f4d4a23b893c20a5f WatchSource:0}: Error finding container c4fdca3b1dca88d18adc5bee04d5b61a9756b6bac1c82a5f4d4a23b893c20a5f: Status 404 returned error can't find the container with id c4fdca3b1dca88d18adc5bee04d5b61a9756b6bac1c82a5f4d4a23b893c20a5f
Jul 11 07:46:26.064376 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:26.064344    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n" event={"ID":"9b049884-7d4d-40ca-a508-1d294895d351","Type":"ContainerStarted","Data":"c4fdca3b1dca88d18adc5bee04d5b61a9756b6bac1c82a5f4d4a23b893c20a5f"}
Jul 11 07:46:27.260310 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:27.260270    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:27.399685 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:27.399646    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:27.669122 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:27.669089    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.008740 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.008617    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.149065 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.149030    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.207581 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.206948    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"]
Jul 11 07:46:28.209925 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.209903    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.217142 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.216262    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-z9sdc\""
Jul 11 07:46:28.217142 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.216486    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 11 07:46:28.217142 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.216683    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 11 07:46:28.217142 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.216840    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 11 07:46:28.217142 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.217020    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 11 07:46:28.221606 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.221479    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"]
Jul 11 07:46:28.244782 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.244747    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.317073 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.317022    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.340086 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.340051    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/237570bd-ba2d-4cc3-ae35-1901addea7e4-tmp\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.340262 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.340098    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tnxc\" (UniqueName: \"kubernetes.io/projected/237570bd-ba2d-4cc3-ae35-1901addea7e4-kube-api-access-8tnxc\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.340366 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.340348    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/237570bd-ba2d-4cc3-ae35-1901addea7e4-tls-certs\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.415611 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.415578    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.441842 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.441795    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/237570bd-ba2d-4cc3-ae35-1901addea7e4-tmp\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.442043 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.441864    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tnxc\" (UniqueName: \"kubernetes.io/projected/237570bd-ba2d-4cc3-ae35-1901addea7e4-kube-api-access-8tnxc\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.442043 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.441927    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/237570bd-ba2d-4cc3-ae35-1901addea7e4-tls-certs\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.444145 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.444120    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/237570bd-ba2d-4cc3-ae35-1901addea7e4-tmp\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.444455 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.444434    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/237570bd-ba2d-4cc3-ae35-1901addea7e4-tls-certs\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.452652 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.452614    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tnxc\" (UniqueName: \"kubernetes.io/projected/237570bd-ba2d-4cc3-ae35-1901addea7e4-kube-api-access-8tnxc\") pod \"kube-auth-proxy-5dd5d4cc95-9vkxw\" (UID: \"237570bd-ba2d-4cc3-ae35-1901addea7e4\") " pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.526917 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.526875    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"
Jul 11 07:46:28.665300 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.665171    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw"]
Jul 11 07:46:28.667716 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:46:28.667690    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237570bd_ba2d_4cc3_ae35_1901addea7e4.slice/crio-f95f51f424957603be58f564e4bdb8fab16131044a913392a588648e40d0204a WatchSource:0}: Error finding container f95f51f424957603be58f564e4bdb8fab16131044a913392a588648e40d0204a: Status 404 returned error can't find the container with id f95f51f424957603be58f564e4bdb8fab16131044a913392a588648e40d0204a
Jul 11 07:46:28.793554 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.793498    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.871687 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.871608    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:28.932840 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:28.932799    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:29.081855 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:29.081821    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw" event={"ID":"237570bd-ba2d-4cc3-ae35-1901addea7e4","Type":"ContainerStarted","Data":"f95f51f424957603be58f564e4bdb8fab16131044a913392a588648e40d0204a"}
Jul 11 07:46:29.083214 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:29.083190    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n" event={"ID":"9b049884-7d4d-40ca-a508-1d294895d351","Type":"ContainerStarted","Data":"99f6139c02482bdc9985a1baa713c0ce7c2680542a48b82d76a50925696c3e6e"}
Jul 11 07:46:29.083348 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:29.083304    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:29.099089 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:29.099035    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n" podStartSLOduration=1.640273483 podStartE2EDuration="4.099014598s" totalImagesPullingTime="2.458741115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:29.098008443 +0000 UTC m=+495.301427305" watchObservedRunningTime="2026-07-11 07:46:29.099014598 +0000 UTC m=+495.302433463"
Jul 11 07:46:29.328131 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:29.328095    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:29.748476 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:29.748388    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:32.098699 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:32.098659    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw" event={"ID":"237570bd-ba2d-4cc3-ae35-1901addea7e4","Type":"ContainerStarted","Data":"ed092a65ff528c6beaf8d05d3d71b48d1d4d0b1a5ee21ce7039087372c69cdd8"}
Jul 11 07:46:32.114547 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:32.114477    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dd5d4cc95-9vkxw" podStartSLOduration=1.482707909 podStartE2EDuration="4.114460439s" totalImagesPullingTime="2.63175253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:32.11345798 +0000 UTC m=+498.316876862" watchObservedRunningTime="2026-07-11 07:46:32.114460439 +0000 UTC m=+498.317879343"
Jul 11 07:46:33.987241 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:33.987212    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:34.041848 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:34.041812    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:34.199183 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:34.199151    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:34.310916 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:34.310870    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:34.662384 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:34.662299    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:37.833449 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.833410    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:37.897476 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.897441    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"]
Jul 11 07:46:37.900894 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.900868    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:37.903372 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.903349    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 11 07:46:37.903641 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.903460    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-h69tp\""
Jul 11 07:46:37.911571 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.911541    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"]
Jul 11 07:46:37.914564 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:37.914515    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:38.022709 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.022674    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:38.025669 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.025647    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z78rp\" (UniqueName: \"kubernetes.io/projected/69e057be-95e0-40e8-bb64-685fafa31d56-kube-api-access-z78rp\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.025743 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.025702    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69e057be-95e0-40e8-bb64-685fafa31d56-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.126798 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.126723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z78rp\" (UniqueName: \"kubernetes.io/projected/69e057be-95e0-40e8-bb64-685fafa31d56-kube-api-access-z78rp\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.126798 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.126773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69e057be-95e0-40e8-bb64-685fafa31d56-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.126988 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:46:38.126867    2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 11 07:46:38.126988 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:46:38.126938    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e057be-95e0-40e8-bb64-685fafa31d56-cert podName:69e057be-95e0-40e8-bb64-685fafa31d56 nodeName:}" failed. No retries permitted until 2026-07-11 07:46:38.626922627 +0000 UTC m=+504.830341469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69e057be-95e0-40e8-bb64-685fafa31d56-cert") pod "llmisvc-controller-manager-777c6bd7d5-gh2gz" (UID: "69e057be-95e0-40e8-bb64-685fafa31d56") : secret "llmisvc-webhook-server-cert" not found
Jul 11 07:46:38.142701 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.142671    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z78rp\" (UniqueName: \"kubernetes.io/projected/69e057be-95e0-40e8-bb64-685fafa31d56-kube-api-access-z78rp\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.307126 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.307092    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:38.631335 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.631305    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69e057be-95e0-40e8-bb64-685fafa31d56-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.633693 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.633675    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69e057be-95e0-40e8-bb64-685fafa31d56-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gh2gz\" (UID: \"69e057be-95e0-40e8-bb64-685fafa31d56\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.814877 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.814833    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:38.943512 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:38.943405    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"]
Jul 11 07:46:38.946126 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:46:38.946098    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod69e057be_95e0_40e8_bb64_685fafa31d56.slice/crio-f933aed5f4db6a7b2365bf59a278dd52897d1c7f1417596cd9cdc7bec49032e0 WatchSource:0}: Error finding container f933aed5f4db6a7b2365bf59a278dd52897d1c7f1417596cd9cdc7bec49032e0: Status 404 returned error can't find the container with id f933aed5f4db6a7b2365bf59a278dd52897d1c7f1417596cd9cdc7bec49032e0
Jul 11 07:46:39.109381 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:39.109347    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:39.132200 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:39.132165    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz" event={"ID":"69e057be-95e0-40e8-bb64-685fafa31d56","Type":"ContainerStarted","Data":"f933aed5f4db6a7b2365bf59a278dd52897d1c7f1417596cd9cdc7bec49032e0"}
Jul 11 07:46:39.254469 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:39.254380    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:39.350887 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:39.350858    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:40.090326 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:40.090301    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-2wm5n"
Jul 11 07:46:42.147437 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:42.147357    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz" event={"ID":"69e057be-95e0-40e8-bb64-685fafa31d56","Type":"ContainerStarted","Data":"ff7c8ba3f59b102dba7207f8eb7f6ab18fbb3f1991b48d712c6aba21828a81a8"}
Jul 11 07:46:42.147799 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:42.147472    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:46:42.161962 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:42.161918    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz" podStartSLOduration=2.290575494 podStartE2EDuration="5.161903432s" totalImagesPullingTime="2.871327938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:42.160898497 +0000 UTC m=+508.364317376" watchObservedRunningTime="2026-07-11 07:46:42.161903432 +0000 UTC m=+508.365322294"
Jul 11 07:46:46.950295 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:46.950259    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:46:47.001716 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:46:47.001660    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:47:07.964204 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:47:07.964120    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:47:13.154681 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:47:13.154637    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gh2gz"
Jul 11 07:47:28.735610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:47:28.735573    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:47:29.994882 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:47:29.994844    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:47:57.048694 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:47:57.048666    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:47:57.630191 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:47:57.630158    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:02.027856 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.027817    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:02.255258 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.255228    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5647f495c5-9fqnd"]
Jul 11 07:48:02.258396 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.258376    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.260648 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.260627    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gqvhv\""
Jul 11 07:48:02.261491 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.261471    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 11 07:48:02.261491 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.261485    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 11 07:48:02.261688 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.261494    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 11 07:48:02.261688 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.261549    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 11 07:48:02.261688 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.261622    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 11 07:48:02.265452 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.265377    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 11 07:48:02.267580 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.267559    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5647f495c5-9fqnd"]
Jul 11 07:48:02.316013 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.315983    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-service-ca\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.316178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.316033    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-console-config\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.316178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.316065    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-oauth-serving-cert\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.316178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.316104    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj4d7\" (UniqueName: \"kubernetes.io/projected/8425ea90-c1bc-4889-a6df-910e8732c627-kube-api-access-qj4d7\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.316178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.316130    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-trusted-ca-bundle\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.316178 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.316154    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8425ea90-c1bc-4889-a6df-910e8732c627-console-serving-cert\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.316357 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.316191    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8425ea90-c1bc-4889-a6df-910e8732c627-console-oauth-config\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.342207 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.342179    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:02.417004 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.416971    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj4d7\" (UniqueName: \"kubernetes.io/projected/8425ea90-c1bc-4889-a6df-910e8732c627-kube-api-access-qj4d7\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417004 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417009    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-trusted-ca-bundle\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417238 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417026    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8425ea90-c1bc-4889-a6df-910e8732c627-console-serving-cert\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417238 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417145    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8425ea90-c1bc-4889-a6df-910e8732c627-console-oauth-config\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417238 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417213    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-service-ca\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417396 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417246    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-console-config\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417396 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417272    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-oauth-serving-cert\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417993 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-oauth-serving-cert\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.417993 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.417990    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-console-config\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.418119 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.418020    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-trusted-ca-bundle\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.418119 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.418038    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8425ea90-c1bc-4889-a6df-910e8732c627-service-ca\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.419425 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.419406    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8425ea90-c1bc-4889-a6df-910e8732c627-console-oauth-config\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.419603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.419588    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8425ea90-c1bc-4889-a6df-910e8732c627-console-serving-cert\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.427781 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.427763    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj4d7\" (UniqueName: \"kubernetes.io/projected/8425ea90-c1bc-4889-a6df-910e8732c627-kube-api-access-qj4d7\") pod \"console-5647f495c5-9fqnd\" (UID: \"8425ea90-c1bc-4889-a6df-910e8732c627\") " pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.437996 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.437973    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:02.567930 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.567845    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:02.697771 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:02.697749    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5647f495c5-9fqnd"]
Jul 11 07:48:02.700346 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:48:02.700317    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8425ea90_c1bc_4889_a6df_910e8732c627.slice/crio-3324bc7a81ad9f2ea365aca8d9591a016b2c40f2f23b33082c67271403ad514f WatchSource:0}: Error finding container 3324bc7a81ad9f2ea365aca8d9591a016b2c40f2f23b33082c67271403ad514f: Status 404 returned error can't find the container with id 3324bc7a81ad9f2ea365aca8d9591a016b2c40f2f23b33082c67271403ad514f
Jul 11 07:48:03.484386 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:03.484355    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5647f495c5-9fqnd" event={"ID":"8425ea90-c1bc-4889-a6df-910e8732c627","Type":"ContainerStarted","Data":"30ae0023c57d684c73fcfb3694247a003e64ea825bd9f617db931b5e0d8df587"}
Jul 11 07:48:03.484807 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:03.484433    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5647f495c5-9fqnd" event={"ID":"8425ea90-c1bc-4889-a6df-910e8732c627","Type":"ContainerStarted","Data":"3324bc7a81ad9f2ea365aca8d9591a016b2c40f2f23b33082c67271403ad514f"}
Jul 11 07:48:03.503087 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:03.503042    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5647f495c5-9fqnd" podStartSLOduration=1.50302736 podStartE2EDuration="1.50302736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:03.501451379 +0000 UTC m=+589.704870242" watchObservedRunningTime="2026-07-11 07:48:03.50302736 +0000 UTC m=+589.706446221"
Jul 11 07:48:04.275755 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:04.275724    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:04.375535 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:04.375486    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:06.897875 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:06.897842    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:07.069487 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:07.069444    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:08.667884 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:08.667851    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:10.236800 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:10.236766    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:11.755494 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:11.755451    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:12.050746 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.050714    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"]
Jul 11 07:48:12.054080 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.054064    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:12.056475 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.056455    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 11 07:48:12.057289 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.057266    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 11 07:48:12.057415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.057301    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 11 07:48:12.057415 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.057318    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sxk74\""
Jul 11 07:48:12.063110 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.063089    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"]
Jul 11 07:48:12.092584 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.092555    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbqlr\" (UniqueName: \"kubernetes.io/projected/0e012a0b-96ff-45dd-85db-426a94966a90-kube-api-access-mbqlr\") pod \"dns-operator-controller-manager-864dd78c4c-q59s4\" (UID: \"0e012a0b-96ff-45dd-85db-426a94966a90\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:12.193604 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.193566    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbqlr\" (UniqueName: \"kubernetes.io/projected/0e012a0b-96ff-45dd-85db-426a94966a90-kube-api-access-mbqlr\") pod \"dns-operator-controller-manager-864dd78c4c-q59s4\" (UID: \"0e012a0b-96ff-45dd-85db-426a94966a90\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:12.201247 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.201215    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbqlr\" (UniqueName: \"kubernetes.io/projected/0e012a0b-96ff-45dd-85db-426a94966a90-kube-api-access-mbqlr\") pod \"dns-operator-controller-manager-864dd78c4c-q59s4\" (UID: \"0e012a0b-96ff-45dd-85db-426a94966a90\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:12.365313 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.365228    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:12.495187 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.495130    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"]
Jul 11 07:48:12.497840 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:48:12.497814    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e012a0b_96ff_45dd_85db_426a94966a90.slice/crio-3b5095f1faac2c4c5f9bc248bb65696875f706f897957694032b4fcf2779f1b7 WatchSource:0}: Error finding container 3b5095f1faac2c4c5f9bc248bb65696875f706f897957694032b4fcf2779f1b7: Status 404 returned error can't find the container with id 3b5095f1faac2c4c5f9bc248bb65696875f706f897957694032b4fcf2779f1b7
Jul 11 07:48:12.526636 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.526608    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4" event={"ID":"0e012a0b-96ff-45dd-85db-426a94966a90","Type":"ContainerStarted","Data":"3b5095f1faac2c4c5f9bc248bb65696875f706f897957694032b4fcf2779f1b7"}
Jul 11 07:48:12.568875 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.568850    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:12.568977 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.568904    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:12.573513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:12.573493    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:13.534240 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:13.534214    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5647f495c5-9fqnd"
Jul 11 07:48:13.647190 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:13.647156    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:13.748300 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:13.748257    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:13.847631 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:13.847598    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:14.033299 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.033265    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:14.094879 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.094842    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:14.174994 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.174976    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 11 07:48:14.213923 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.213898    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:48:14.217814 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.217795    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:48:14.535956 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.535869    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4" event={"ID":"0e012a0b-96ff-45dd-85db-426a94966a90","Type":"ContainerStarted","Data":"55f4db028d9cd7c5b1872666205382dc4faf654899f704acbc8d3b016ef44f5e"}
Jul 11 07:48:14.536322 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.535997    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:14.552610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:14.552557    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4" podStartSLOduration=0.879666758 podStartE2EDuration="2.552544066s" totalImagesPullingTime="1.672877308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:14.550577583 +0000 UTC m=+600.753996445" watchObservedRunningTime="2026-07-11 07:48:14.552544066 +0000 UTC m=+600.755962920"
Jul 11 07:48:15.084539 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.084502    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"]
Jul 11 07:48:15.087447 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.087430    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.089992 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.089973    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 11 07:48:15.090113 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.089997    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 11 07:48:15.090230 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.090216    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j6pgs\""
Jul 11 07:48:15.094366 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.094345    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"]
Jul 11 07:48:15.098520 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.098495    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:15.118809 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.118789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff579be5-8da7-4a10-adfe-e654d709ab3d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.118908 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.118816    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff579be5-8da7-4a10-adfe-e654d709ab3d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.118908 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.118870    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnxqt\" (UniqueName: \"kubernetes.io/projected/ff579be5-8da7-4a10-adfe-e654d709ab3d-kube-api-access-tnxqt\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.219605 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.219576    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnxqt\" (UniqueName: \"kubernetes.io/projected/ff579be5-8da7-4a10-adfe-e654d709ab3d-kube-api-access-tnxqt\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.219733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.219621    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff579be5-8da7-4a10-adfe-e654d709ab3d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.219733 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.219648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff579be5-8da7-4a10-adfe-e654d709ab3d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.220032 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:15.219743    2572 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 11 07:48:15.220032 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:15.219801    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff579be5-8da7-4a10-adfe-e654d709ab3d-plugin-serving-cert podName:ff579be5-8da7-4a10-adfe-e654d709ab3d nodeName:}" failed. No retries permitted until 2026-07-11 07:48:15.719783023 +0000 UTC m=+601.923201866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/ff579be5-8da7-4a10-adfe-e654d709ab3d-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-z69bp" (UID: "ff579be5-8da7-4a10-adfe-e654d709ab3d") : secret "plugin-serving-cert" not found
Jul 11 07:48:15.220331 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.220309    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff579be5-8da7-4a10-adfe-e654d709ab3d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.228234 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.228209    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnxqt\" (UniqueName: \"kubernetes.io/projected/ff579be5-8da7-4a10-adfe-e654d709ab3d-kube-api-access-tnxqt\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.305832 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.305803    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:15.723211 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.723184    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff579be5-8da7-4a10-adfe-e654d709ab3d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:15.725537 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:15.725507    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff579be5-8da7-4a10-adfe-e654d709ab3d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z69bp\" (UID: \"ff579be5-8da7-4a10-adfe-e654d709ab3d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:16.006197 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:16.006111    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"
Jul 11 07:48:16.133348 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:16.133306    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp"]
Jul 11 07:48:16.136580 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:48:16.136521    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff579be5_8da7_4a10_adfe_e654d709ab3d.slice/crio-bdf67ec53d519c7aeaf0f3a4c1831d843c27b5e957be4d13d5332add1cf22bf7 WatchSource:0}: Error finding container bdf67ec53d519c7aeaf0f3a4c1831d843c27b5e957be4d13d5332add1cf22bf7: Status 404 returned error can't find the container with id bdf67ec53d519c7aeaf0f3a4c1831d843c27b5e957be4d13d5332add1cf22bf7
Jul 11 07:48:16.546223 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:16.546182    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp" event={"ID":"ff579be5-8da7-4a10-adfe-e654d709ab3d","Type":"ContainerStarted","Data":"bdf67ec53d519c7aeaf0f3a4c1831d843c27b5e957be4d13d5332add1cf22bf7"}
Jul 11 07:48:18.239922 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:18.239857    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:18.291231 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:18.291171    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:18.636943 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:18.636865    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:18.743306 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:18.743267    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:24.939784 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:24.939745    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:25.074331 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.074292    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:25.180208 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.180133    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:25.291177 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.291101    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"]
Jul 11 07:48:25.314280 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.314249    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"]
Jul 11 07:48:25.314452 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.314345    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.316718 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.316683    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f5qqf\""
Jul 11 07:48:25.413886 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.413844    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgl6c\" (UniqueName: \"kubernetes.io/projected/189bbd24-003c-47bf-8076-efe763c7fd36-kube-api-access-cgl6c\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.414078 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.413936    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/189bbd24-003c-47bf-8076-efe763c7fd36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.514762 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.514724    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/189bbd24-003c-47bf-8076-efe763c7fd36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.514947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.514815    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgl6c\" (UniqueName: \"kubernetes.io/projected/189bbd24-003c-47bf-8076-efe763c7fd36-kube-api-access-cgl6c\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.515134 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.515111    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/189bbd24-003c-47bf-8076-efe763c7fd36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.535222 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.535163    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgl6c\" (UniqueName: \"kubernetes.io/projected/189bbd24-003c-47bf-8076-efe763c7fd36-kube-api-access-cgl6c\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.543383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.543103    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q59s4"
Jul 11 07:48:25.555045 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.555016    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"]
Jul 11 07:48:25.555346 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.555323    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:25.881521 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.881488    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"]
Jul 11 07:48:25.898612 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.898541    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"]
Jul 11 07:48:25.973756 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.973725    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"]
Jul 11 07:48:25.982770 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.982745    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:25.985651 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.985629    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-95wdx\""
Jul 11 07:48:25.991400 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:25.991111    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"]
Jul 11 07:48:26.120544 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:26.120501    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbxg5\" (UniqueName: \"kubernetes.io/projected/dd5bbddd-f698-4b78-9bed-599d6e399040-kube-api-access-bbxg5\") pod \"limitador-operator-controller-manager-859d7b78bd-kz64m\" (UID: \"dd5bbddd-f698-4b78-9bed-599d6e399040\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:26.221274 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:26.221177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbxg5\" (UniqueName: \"kubernetes.io/projected/dd5bbddd-f698-4b78-9bed-599d6e399040-kube-api-access-bbxg5\") pod \"limitador-operator-controller-manager-859d7b78bd-kz64m\" (UID: \"dd5bbddd-f698-4b78-9bed-599d6e399040\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:26.231239 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:26.230928    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbxg5\" (UniqueName: \"kubernetes.io/projected/dd5bbddd-f698-4b78-9bed-599d6e399040-kube-api-access-bbxg5\") pod \"limitador-operator-controller-manager-859d7b78bd-kz64m\" (UID: \"dd5bbddd-f698-4b78-9bed-599d6e399040\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:26.297114 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:26.297077    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:28.374710 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:28.374674    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:28.475161 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:28.475117    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:33.759168 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:33.759127    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:33.827513 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:33.827476    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:34.356572 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:34.356515    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:34.413484 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:34.413441    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:37.519314 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:37.519290    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"]
Jul 11 07:48:37.522431 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:48:37.522404    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5bbddd_f698_4b78_9bed_599d6e399040.slice/crio-8b46839a8f8c87be42f6fcc10d231f525709b8db2adaff3b28f1d9ac6c64e3f7 WatchSource:0}: Error finding container 8b46839a8f8c87be42f6fcc10d231f525709b8db2adaff3b28f1d9ac6c64e3f7: Status 404 returned error can't find the container with id 8b46839a8f8c87be42f6fcc10d231f525709b8db2adaff3b28f1d9ac6c64e3f7
Jul 11 07:48:37.656175 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:37.656097    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m" event={"ID":"dd5bbddd-f698-4b78-9bed-599d6e399040","Type":"ContainerStarted","Data":"8b46839a8f8c87be42f6fcc10d231f525709b8db2adaff3b28f1d9ac6c64e3f7"}
Jul 11 07:48:38.166683 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:38.166647    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:38.267402 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:38.267373    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:38.663448 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:38.663409    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp" event={"ID":"ff579be5-8da7-4a10-adfe-e654d709ab3d","Type":"ContainerStarted","Data":"368eda7c988a07f3277ff29db75583529083578f89b0e8ece4204ff8cf942fad"}
Jul 11 07:48:38.692313 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:38.692267    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z69bp" podStartSLOduration=2.064274426 podStartE2EDuration="23.692251328s" totalImagesPullingTime="21.627976902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:38.690165636 +0000 UTC m=+624.893584498" watchObservedRunningTime="2026-07-11 07:48:38.692251328 +0000 UTC m=+624.895670190"
Jul 11 07:48:40.391126 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:40.391087    2572 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-9vrhw_kuadrant-system_189bbd24-003c-47bf-8076-efe763c7fd36_0(92050acf420eeee324e2da225cc2ffbe82bea4438bcce205522fa25eb14bde5d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-9vrhw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 11 07:48:40.391501 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:40.391152    2572 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-9vrhw_kuadrant-system_189bbd24-003c-47bf-8076-efe763c7fd36_0(92050acf420eeee324e2da225cc2ffbe82bea4438bcce205522fa25eb14bde5d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-9vrhw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:40.673115 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.673035    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:40.675230 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:40.675205    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" is forbidden: User \"system:node:ip-10-0-141-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-181.ec2.internal' and this object" podUID="189bbd24-003c-47bf-8076-efe763c7fd36" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:40.689293 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.689270    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:40.691229 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:40.691207    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" is forbidden: User \"system:node:ip-10-0-141-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-181.ec2.internal' and this object" podUID="189bbd24-003c-47bf-8076-efe763c7fd36" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:40.859349 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.859326    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/189bbd24-003c-47bf-8076-efe763c7fd36-extensions-socket-volume\") pod \"189bbd24-003c-47bf-8076-efe763c7fd36\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") "
Jul 11 07:48:40.859449 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.859390    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgl6c\" (UniqueName: \"kubernetes.io/projected/189bbd24-003c-47bf-8076-efe763c7fd36-kube-api-access-cgl6c\") pod \"189bbd24-003c-47bf-8076-efe763c7fd36\" (UID: \"189bbd24-003c-47bf-8076-efe763c7fd36\") "
Jul 11 07:48:40.859590 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.859569    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189bbd24-003c-47bf-8076-efe763c7fd36-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "189bbd24-003c-47bf-8076-efe763c7fd36" (UID: "189bbd24-003c-47bf-8076-efe763c7fd36"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 07:48:40.861266 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.861246    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189bbd24-003c-47bf-8076-efe763c7fd36-kube-api-access-cgl6c" (OuterVolumeSpecName: "kube-api-access-cgl6c") pod "189bbd24-003c-47bf-8076-efe763c7fd36" (UID: "189bbd24-003c-47bf-8076-efe763c7fd36"). InnerVolumeSpecName "kube-api-access-cgl6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:48:40.960644 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.960562    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/189bbd24-003c-47bf-8076-efe763c7fd36-extensions-socket-volume\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:48:40.960644 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:40.960590    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgl6c\" (UniqueName: \"kubernetes.io/projected/189bbd24-003c-47bf-8076-efe763c7fd36-kube-api-access-cgl6c\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:48:41.678356 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:41.678328    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:41.678838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:41.678362    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m" event={"ID":"dd5bbddd-f698-4b78-9bed-599d6e399040","Type":"ContainerStarted","Data":"d94f210a869436a547dc45745698299e5267e4bf551d99ac9f77f9c7bd0a3f08"}
Jul 11 07:48:41.678838 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:41.678503    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:41.680274 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:41.680241    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" is forbidden: User \"system:node:ip-10-0-141-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-181.ec2.internal' and this object" podUID="189bbd24-003c-47bf-8076-efe763c7fd36" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:41.700829 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:41.700781    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m" podStartSLOduration=13.486222387 podStartE2EDuration="16.700760336s" totalImagesPullingTime="3.214537949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:41.699060793 +0000 UTC m=+627.902479674" watchObservedRunningTime="2026-07-11 07:48:41.700760336 +0000 UTC m=+627.904179264"
Jul 11 07:48:41.701141 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:48:41.701116    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vrhw\" is forbidden: User \"system:node:ip-10-0-141-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-181.ec2.internal' and this object" podUID="189bbd24-003c-47bf-8076-efe763c7fd36" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vrhw"
Jul 11 07:48:42.294899 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:42.294866    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189bbd24-003c-47bf-8076-efe763c7fd36" path="/var/lib/kubelet/pods/189bbd24-003c-47bf-8076-efe763c7fd36/volumes"
Jul 11 07:48:44.583302 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:44.583256    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:44.683177 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:44.683130    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:47.010893 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:47.010862    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:47.111774 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:47.111746    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:52.684594 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:52.684566    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kz64m"
Jul 11 07:48:57.992792 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:57.992759    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:48:58.344982 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:48:58.344946    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:11.350212 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:11.350178    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:11.450023 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:11.449988    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:11.500850 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:11.500816    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:12.130381 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.130353    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-gdfvn"]
Jul 11 07:49:12.132647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.132629    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:12.135057 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.135035    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h5kjn\""
Jul 11 07:49:12.144803 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.144777    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gdfvn"]
Jul 11 07:49:12.160260 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.160230    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:12.207555 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.207488    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn44l\" (UniqueName: \"kubernetes.io/projected/703ba6a3-290c-4ab5-b26b-a0ba02e713ca-kube-api-access-mn44l\") pod \"authorino-58cd955675-gdfvn\" (UID: \"703ba6a3-290c-4ab5-b26b-a0ba02e713ca\") " pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:12.261103 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.261073    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:12.308951 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.308919    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn44l\" (UniqueName: \"kubernetes.io/projected/703ba6a3-290c-4ab5-b26b-a0ba02e713ca-kube-api-access-mn44l\") pod \"authorino-58cd955675-gdfvn\" (UID: \"703ba6a3-290c-4ab5-b26b-a0ba02e713ca\") " pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:12.316208 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.316179    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn44l\" (UniqueName: \"kubernetes.io/projected/703ba6a3-290c-4ab5-b26b-a0ba02e713ca-kube-api-access-mn44l\") pod \"authorino-58cd955675-gdfvn\" (UID: \"703ba6a3-290c-4ab5-b26b-a0ba02e713ca\") " pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:12.441205 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.441121    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:12.567184 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.567143    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gdfvn"]
Jul 11 07:49:12.569751 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:49:12.569711    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod703ba6a3_290c_4ab5_b26b_a0ba02e713ca.slice/crio-5add258b69fffe6e3cd3071417c13883cb9d5ae4652dea0dc7e1e70bdc11da19 WatchSource:0}: Error finding container 5add258b69fffe6e3cd3071417c13883cb9d5ae4652dea0dc7e1e70bdc11da19: Status 404 returned error can't find the container with id 5add258b69fffe6e3cd3071417c13883cb9d5ae4652dea0dc7e1e70bdc11da19
Jul 11 07:49:12.817610 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:12.817580    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gdfvn" event={"ID":"703ba6a3-290c-4ab5-b26b-a0ba02e713ca","Type":"ContainerStarted","Data":"5add258b69fffe6e3cd3071417c13883cb9d5ae4652dea0dc7e1e70bdc11da19"}
Jul 11 07:49:15.837788 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:15.837743    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gdfvn" event={"ID":"703ba6a3-290c-4ab5-b26b-a0ba02e713ca","Type":"ContainerStarted","Data":"e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb"}
Jul 11 07:49:15.853912 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:15.853844    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-gdfvn" podStartSLOduration=0.906985126 podStartE2EDuration="3.853827937s" totalImagesPullingTime="2.946842811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:49:15.852839748 +0000 UTC m=+662.056258609" watchObservedRunningTime="2026-07-11 07:49:15.853827937 +0000 UTC m=+662.057246799"
Jul 11 07:49:17.608979 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:17.608946    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gdfvn"]
Jul 11 07:49:17.846778 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:17.846741    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-gdfvn" podUID="703ba6a3-290c-4ab5-b26b-a0ba02e713ca" containerName="authorino" containerID="cri-o://e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb" gracePeriod=30
Jul 11 07:49:18.095540 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.095506    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:18.162944 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.162868    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn44l\" (UniqueName: \"kubernetes.io/projected/703ba6a3-290c-4ab5-b26b-a0ba02e713ca-kube-api-access-mn44l\") pod \"703ba6a3-290c-4ab5-b26b-a0ba02e713ca\" (UID: \"703ba6a3-290c-4ab5-b26b-a0ba02e713ca\") "
Jul 11 07:49:18.164939 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.164919    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703ba6a3-290c-4ab5-b26b-a0ba02e713ca-kube-api-access-mn44l" (OuterVolumeSpecName: "kube-api-access-mn44l") pod "703ba6a3-290c-4ab5-b26b-a0ba02e713ca" (UID: "703ba6a3-290c-4ab5-b26b-a0ba02e713ca"). InnerVolumeSpecName "kube-api-access-mn44l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 07:49:18.264319 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.264285    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn44l\" (UniqueName: \"kubernetes.io/projected/703ba6a3-290c-4ab5-b26b-a0ba02e713ca-kube-api-access-mn44l\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 07:49:18.851869 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.851832    2572 generic.go:356] "Generic (PLEG): container finished" podID="703ba6a3-290c-4ab5-b26b-a0ba02e713ca" containerID="e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb" exitCode=0
Jul 11 07:49:18.852319 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.851873    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gdfvn"
Jul 11 07:49:18.852319 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.851905    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gdfvn" event={"ID":"703ba6a3-290c-4ab5-b26b-a0ba02e713ca","Type":"ContainerDied","Data":"e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb"}
Jul 11 07:49:18.852319 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.851940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gdfvn" event={"ID":"703ba6a3-290c-4ab5-b26b-a0ba02e713ca","Type":"ContainerDied","Data":"5add258b69fffe6e3cd3071417c13883cb9d5ae4652dea0dc7e1e70bdc11da19"}
Jul 11 07:49:18.852319 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.851959    2572 scope.go:117] "RemoveContainer" containerID="e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb"
Jul 11 07:49:18.859898 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.859879    2572 scope.go:117] "RemoveContainer" containerID="e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb"
Jul 11 07:49:18.860185 ip-10-0-141-181 kubenswrapper[2572]: E0711 07:49:18.860160    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb\": container with ID starting with e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb not found: ID does not exist" containerID="e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb"
Jul 11 07:49:18.860248 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.860192    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb"} err="failed to get container status \"e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb\": rpc error: code = NotFound desc = could not find container \"e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb\": container with ID starting with e936449ccce84eddbdd359c64c9f02f543608575761fdbc7f146cd34b8e36eeb not found: ID does not exist"
Jul 11 07:49:18.865097 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.865076    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gdfvn"]
Jul 11 07:49:18.867519 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:18.867501    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-gdfvn"]
Jul 11 07:49:20.294416 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:20.294380    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="703ba6a3-290c-4ab5-b26b-a0ba02e713ca" path="/var/lib/kubelet/pods/703ba6a3-290c-4ab5-b26b-a0ba02e713ca/volumes"
Jul 11 07:49:31.134141 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:31.134100    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:31.236125 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:31.236085    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:31.608986 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:31.608931    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:43.252209 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:43.252172    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:43.776537 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:43.776497    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:50.293479 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:50.293445    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:50.367573 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:50.367535    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:50.416545 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:50.416504    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:53.156054 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:53.155983    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:49:53.255405 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:49:53.255369    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:24.779111 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:24.779065    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:24.876638 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:24.876582    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:25.339095 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:25.339054    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:25.642774 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:25.642682    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:25.752728 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:25.752695    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:25.848331 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:25.848294    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:37.153330 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:37.153287    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:37.252454 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:37.252423    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:40.857951 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.857920    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"]
Jul 11 07:50:40.858442 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.858369    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="703ba6a3-290c-4ab5-b26b-a0ba02e713ca" containerName="authorino"
Jul 11 07:50:40.858442 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.858387    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="703ba6a3-290c-4ab5-b26b-a0ba02e713ca" containerName="authorino"
Jul 11 07:50:40.858603 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.858466    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="703ba6a3-290c-4ab5-b26b-a0ba02e713ca" containerName="authorino"
Jul 11 07:50:40.861441 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.861418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.863844 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.863816    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 11 07:50:40.863982 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.863817    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 11 07:50:40.863982 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.863871    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 11 07:50:40.864623 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.864604    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wldk6\""
Jul 11 07:50:40.870931 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.870907    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"]
Jul 11 07:50:40.890279 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.890248    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:40.893305 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.893284    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvgd9\" (UniqueName: \"kubernetes.io/projected/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-kube-api-access-vvgd9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.893395 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.893317    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.893395 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.893335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.893395 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.893361    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.893495 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.893422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.893545 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.893500    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.994612 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.994573    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.994612 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.994615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvgd9\" (UniqueName: \"kubernetes.io/projected/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-kube-api-access-vvgd9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.994864 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.994737    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.994864 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.994766    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.994864 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.994787    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.994864 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.994825    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.995181 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.995161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.995246 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.995189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.995246 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.995216    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.996795 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.996766    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:40.997161 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:40.997143    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:41.001516 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:41.001491    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvgd9\" (UniqueName: \"kubernetes.io/projected/22b726d0-21a7-4cbf-8a42-d50b8b53a0f0-kube-api-access-vvgd9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99\" (UID: \"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:41.173251 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:41.173152    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:41.305712 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:41.305684    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"]
Jul 11 07:50:41.308382 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:50:41.308354    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b726d0_21a7_4cbf_8a42_d50b8b53a0f0.slice/crio-bfba59475ac3f21f89d7c0905062aa0073017b33fefea64da02f75886ef94685 WatchSource:0}: Error finding container bfba59475ac3f21f89d7c0905062aa0073017b33fefea64da02f75886ef94685: Status 404 returned error can't find the container with id bfba59475ac3f21f89d7c0905062aa0073017b33fefea64da02f75886ef94685
Jul 11 07:50:41.309749 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:41.309734    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 07:50:41.908273 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:41.908184    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:50:42.245947 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:42.245847    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99" event={"ID":"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0","Type":"ContainerStarted","Data":"bfba59475ac3f21f89d7c0905062aa0073017b33fefea64da02f75886ef94685"}
Jul 11 07:50:43.252032 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:43.251995    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99" event={"ID":"22b726d0-21a7-4cbf-8a42-d50b8b53a0f0","Type":"ContainerStarted","Data":"9eb0a35d3981e9cd167aef3e018d6602164405e10e9e5a0f075886caf7270f32"}
Jul 11 07:50:43.269169 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:43.269114    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99" podStartSLOduration=2.169255105 podStartE2EDuration="3.26909783s" totalImagesPullingTime="1.099842725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:50:43.267387686 +0000 UTC m=+749.470806572" watchObservedRunningTime="2026-07-11 07:50:43.26909783 +0000 UTC m=+749.472516692"
Jul 11 07:50:51.173709 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:51.173664    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:51.174143 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:51.173737    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:51.186064 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:51.186031    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:51.301076 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:51.301043    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99"
Jul 11 07:50:54.276042 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:50:54.276013    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:11.157191 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.157157    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"]
Jul 11 07:51:11.160482 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.160464    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.162719 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.162703    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 11 07:51:11.170151 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.170127    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"]
Jul 11 07:51:11.182514 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.182470    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.182664 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.182568    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.182664 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.182602    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.182664 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.182628    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.182832 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.182709    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkwpd\" (UniqueName: \"kubernetes.io/projected/09f906ab-adce-44de-a9c7-ca67bbeba4cf-kube-api-access-nkwpd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.182832 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.182733    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f906ab-adce-44de-a9c7-ca67bbeba4cf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.248543 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.248491    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:11.283253 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283222    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283308    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283338    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkwpd\" (UniqueName: \"kubernetes.io/projected/09f906ab-adce-44de-a9c7-ca67bbeba4cf-kube-api-access-nkwpd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283383 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283355    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f906ab-adce-44de-a9c7-ca67bbeba4cf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283659 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283630    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283714 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.283793 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.283773    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.285555 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.285516    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f906ab-adce-44de-a9c7-ca67bbeba4cf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.285803 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.285788    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f906ab-adce-44de-a9c7-ca67bbeba4cf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.290459 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.290439    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkwpd\" (UniqueName: \"kubernetes.io/projected/09f906ab-adce-44de-a9c7-ca67bbeba4cf-kube-api-access-nkwpd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ps6rr\" (UID: \"09f906ab-adce-44de-a9c7-ca67bbeba4cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.470456 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.470375    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:11.597866 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:11.597837    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"]
Jul 11 07:51:11.600844 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:51:11.600818    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f906ab_adce_44de_a9c7_ca67bbeba4cf.slice/crio-e0329d5c58e733d1de60904dbf20f77de07376a1e58a74a824c102575f6810d4 WatchSource:0}: Error finding container e0329d5c58e733d1de60904dbf20f77de07376a1e58a74a824c102575f6810d4: Status 404 returned error can't find the container with id e0329d5c58e733d1de60904dbf20f77de07376a1e58a74a824c102575f6810d4
Jul 11 07:51:12.389483 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:12.389436    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr" event={"ID":"09f906ab-adce-44de-a9c7-ca67bbeba4cf","Type":"ContainerStarted","Data":"e58dfbd99a00bac470bba8cdc70db4d4f15bc810e6beb72f2475c077ec29878b"}
Jul 11 07:51:12.389881 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:12.389495    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr" event={"ID":"09f906ab-adce-44de-a9c7-ca67bbeba4cf","Type":"ContainerStarted","Data":"e0329d5c58e733d1de60904dbf20f77de07376a1e58a74a824c102575f6810d4"}
Jul 11 07:51:12.405498 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:12.405443    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr" podStartSLOduration=1.237804288 podStartE2EDuration="1.405426822s" totalImagesPullingTime="167.622534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:51:12.403512328 +0000 UTC m=+778.606931191" watchObservedRunningTime="2026-07-11 07:51:12.405426822 +0000 UTC m=+778.608845685"
Jul 11 07:51:12.658059 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:12.657974    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:12.761806 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:12.761752    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:21.471418 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:21.471387    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:21.471862 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:21.471476    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:21.484016 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:21.483992    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:22.448045 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:22.448018    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ps6rr"
Jul 11 07:51:27.844109 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:27.844073    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:40.082703 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:40.082611    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:50.052850 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.052812    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"]
Jul 11 07:51:50.056390 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.056374    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.058478 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.058454    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 11 07:51:50.064301 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.064276    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"]
Jul 11 07:51:50.073904 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.073874    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:51:50.246887 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.246844    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.247085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.246936    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.247085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.246973    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.247085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.247000    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.247085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.247019    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7rzn\" (UniqueName: \"kubernetes.io/projected/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-kube-api-access-f7rzn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.247085 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.247058    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.347789 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.347689    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.347789 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.347741    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.347789 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.347766    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.348069 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.347881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7rzn\" (UniqueName: \"kubernetes.io/projected/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-kube-api-access-f7rzn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.348069 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.347921    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.348069 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.347985    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.348242 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.348105    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.348242 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.348150    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.348242 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.348203    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.350302 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.350280    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.350414 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.350399    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.354938 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.354920    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7rzn\" (UniqueName: \"kubernetes.io/projected/5b7d2e40-9534-402f-8802-f5aacd7ae8f7-kube-api-access-f7rzn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx\" (UID: \"5b7d2e40-9534-402f-8802-f5aacd7ae8f7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.365956 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.365934    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:51:50.510246 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.510219    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"]
Jul 11 07:51:50.512635 ip-10-0-141-181 kubenswrapper[2572]: W0711 07:51:50.512593    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b7d2e40_9534_402f_8802_f5aacd7ae8f7.slice/crio-f8ab9bbc29466b32d32ec9342ace97d5f6313bdc48b5be1c601a5a2e69e3689a WatchSource:0}: Error finding container f8ab9bbc29466b32d32ec9342ace97d5f6313bdc48b5be1c601a5a2e69e3689a: Status 404 returned error can't find the container with id f8ab9bbc29466b32d32ec9342ace97d5f6313bdc48b5be1c601a5a2e69e3689a
Jul 11 07:51:50.573604 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:50.573568    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx" event={"ID":"5b7d2e40-9534-402f-8802-f5aacd7ae8f7","Type":"ContainerStarted","Data":"f8ab9bbc29466b32d32ec9342ace97d5f6313bdc48b5be1c601a5a2e69e3689a"}
Jul 11 07:51:51.579801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:51.579763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx" event={"ID":"5b7d2e40-9534-402f-8802-f5aacd7ae8f7","Type":"ContainerStarted","Data":"56b957ee63a781c33988d5082503a984a3b49a34218483152daac77afe518308"}
Jul 11 07:51:51.595711 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:51:51.595663    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx" podStartSLOduration=1.420518336 podStartE2EDuration="1.595646226s" totalImagesPullingTime="175.12789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:51:51.594649511 +0000 UTC m=+817.798068392" watchObservedRunningTime="2026-07-11 07:51:51.595646226 +0000 UTC m=+817.799065088"
Jul 11 07:52:00.366658 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:00.366610    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:52:00.366658 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:00.366670    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:52:00.378928 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:00.378902    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:52:00.635403 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:00.635319    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx"
Jul 11 07:52:10.006298 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:10.006257    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:52:10.055429 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:10.055391    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:52:27.850575 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:52:27.850504    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:53:12.352241 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:53:12.352202    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:53:13.313296 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:53:13.313261    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:53:13.426823 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:53:13.426791    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:53:13.517766 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:53:13.517730    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:53:14.249154 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:53:14.249122    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:53:14.251801 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:53:14.251778    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:54:23.016501 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:54:23.016464    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:54:24.901822 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:54:24.901787    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:03.216072 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:03.215970    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:03.319413 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:03.319374    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:03.447647 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:03.447612    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:08.980673 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:08.980626    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:09.081897 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:09.081859    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:09.180358 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:09.180325    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:09.317700 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:09.317659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:14.832829 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:14.832796    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:55:14.893669 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:55:14.893636    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:56:00.965899 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:56:00.965864    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:56:18.985304 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:56:18.985210    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:56:19.034618 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:56:19.034585    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:56:26.912993 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:56:26.912955    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:58:00.541968 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:58:00.541894    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:58:00.631058 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:58:00.631025    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 07:58:14.279916 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:58:14.279889    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 07:58:14.283220 ip-10-0-141-181 kubenswrapper[2572]: I0711 07:58:14.283197    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:03:14.307865 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:03:14.307837    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:03:14.316819 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:03:14.316791    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:04:55.727612 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:04:55.727573    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:05:20.199287 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:05:20.199200    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:05:41.513876 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:05:41.513845    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:02.474295 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:02.474259    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:05.382646 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:05.382614    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:05.481301 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:05.481269    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:05.851487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:05.851455    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:11.409576 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:11.409522    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:11.469762 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:11.469735    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:11.611645 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:11.611609    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:11.670768 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:11.670691    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:12.710036 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:12.710003    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:12.790843 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:12.790810    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:12.854982 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:12.854952    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:13.148893 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:13.148861    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:13.256752 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:13.256722    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:13.358563 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:13.358514    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:06:15.266195 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:06:15.266164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:18.206224 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:18.206194    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:22.904073 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:22.904039    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:28.523881 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:28.523852    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:28.584055 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:28.584019    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:33.085302 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:33.085269    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:33.447080 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:33.446993    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:33.560039 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:33.560007    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:33.708947 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:33.708866    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:34.954047 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:34.954012    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:35.061085 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:35.061054    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:35.170143 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:35.170116    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:35.525166 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:35.525137    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:35.625851 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:35.625819    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:39.555128 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:39.555100    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:39.608374 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:39.608346    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:07:42.798315 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:07:42.798232    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:14.336498 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:14.336471    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:08:14.353031 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:14.353004    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:08:39.523679 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:39.523644    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:39.627249 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:39.627217    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:44.854818 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:44.854788    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:45.697587 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:45.697552    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:45.798160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:45.798123    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:51.119767 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:51.119733    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:51.220129 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:51.220099    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:51.327515 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:51.327474    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:51.427405 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:51.427307    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:52.445451 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:52.445412    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:52.529913 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:52.529873    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:52.586864 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:52.586831    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:52.701746 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:52.701672    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:53.183817 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:53.183784    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:53.284206 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:53.284174    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:08:54.822838 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:08:54.822802    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:09:45.695661 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:09:45.695568    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:09:45.796505 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:09:45.796463    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:09:58.213616 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:09:58.213576    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:09:59.234712 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:09:59.234667    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:07.980368 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:07.980326    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:12.688459 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:12.688428    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:18.372577 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:18.372549    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:18.430034 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:18.430006    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:23.692239 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:23.692199    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:23.973023 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:23.972983    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:25.084188 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:25.084156    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:25.150055 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:25.150026    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:25.265061 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:25.265030    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:25.363442 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:25.363357    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:25.815823 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:25.815793    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:25.916786 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:25.916755    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:35.383345 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:35.383303    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:35.448047 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:35.448015    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:10:39.160559 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:10:39.160509    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:29.924798 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:29.924766    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:34.578564 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:34.578510    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:40.515092 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:40.515057    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:40.615475 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:40.615442    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:44.764168 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:44.764136    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:45.340009 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:45.339970    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:45.439315 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:45.439282    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:45.538981 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:45.538942    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:46.280388 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:46.280348    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:46.971805 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:46.971750    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:51.160164 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:51.160128    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:52.815866 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:52.815830    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:52.934036 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:52.934004    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:53.025628 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:53.025598    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:53.123228 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:53.123156    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:53.546176 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:53.546097    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:53.724286 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:53.724253    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:11:53.825225 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:11:53.825188    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:19.502914 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:19.502878    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:21.727980 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:21.727942    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:25.686237 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:25.686206    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:25.789853 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:25.789815    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:25.895039 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:25.895008    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:41.761080 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:41.760998    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:47.525647 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:47.525614    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:47.630425 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:47.630395    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:47.725459 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:47.725429    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:58.256394 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:58.256357    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:12:59.050423 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:12:59.050393    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:04.822130 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:04.822093    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:04.882700 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:04.882672    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:04.950299 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:04.950252    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:05.049874 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:05.049833    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:05.150346 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:05.150263    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:05.864023 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:05.863982    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:05.962485 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:05.962456    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:06.062792 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:06.062761    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:07.679834 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:07.679802    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:13.690417 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:13.690383    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:13.761420 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:13.761391    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:14.346284 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:14.346257    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:14.371668 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:14.371647    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:13:14.381490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:14.381465    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:13:20.440936 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:20.440908    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:20.508358 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:20.508334    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:20.662541 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:20.662507    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:21.079795 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:21.079764    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:21.274034 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:21.274000    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:24.259894 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:24.259860    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:24.335689 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:24.335660    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:24.408634 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:24.408599    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:24.547672 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:24.547587    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:24.799691 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:24.799665    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:24.900097 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:24.900066    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:28.751119 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:28.751088    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:29.412031 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:29.412002    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:29.500377 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:29.500346    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:29.813766 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:29.813735    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:35.499381 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:35.499351    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:35.598987 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:35.598948    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:46.316942 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:46.316889    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:50.942921 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:50.942894    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:56.494366 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:56.494341    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:13:56.595629 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:13:56.595597    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:01.580167 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:01.580134    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:01.806966 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:01.806936    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:01.920192 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:01.920088    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:02.018209 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:02.018182    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:02.135079 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:02.135047    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:07.758701 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:07.758625    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:07.819309 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:07.819277    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:07.894473 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:07.894445    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:07.995612 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:07.995578    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:08.103669 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:08.103640    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:13.814685 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:13.814651    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:13.880502 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:13.880470    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:20.502201 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:20.502165    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:20.782755 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:20.782670    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:21.445440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:21.445410    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:23.528504 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:23.528474    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:23.608302 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:23.608265    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:23.671763 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:23.671726    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:23.774483 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:23.774447    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:24.087592 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:24.087552    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:24.191984 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:24.191950    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:24.632071 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:24.632039    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:28.019317 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:28.019285    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:37.447157 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.447115    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"]
Jul 11 08:14:37.450869 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.450845    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.453404 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.453380    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-683f5ca7\"/\"kube-root-ca.crt\""
Jul 11 08:14:37.453404 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.453392    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-683f5ca7\"/\"openshift-service-ca.crt\""
Jul 11 08:14:37.454363 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.454345    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-683f5ca7\"/\"default-dockercfg-7x987\""
Jul 11 08:14:37.454680 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.454355    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-683f5ca7\"/\"e2e-lifecycle-model-683f5ca7-kserve-self-signed-certs\""
Jul 11 08:14:37.459852 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.459831    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"]
Jul 11 08:14:37.477186 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.477163    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:37.566766 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.566738    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-model-cache\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.566936 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.566802    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-home\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.566936 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.566831    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8cnn\" (UniqueName: \"kubernetes.io/projected/46baf25b-4f24-4202-9870-b45a7ab4e787-kube-api-access-g8cnn\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.566936 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.566873    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-dshm\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.567051 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.566954    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46baf25b-4f24-4202-9870-b45a7ab4e787-tls-certs\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.567051 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.566999    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-tmp-dir\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668238    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-dshm\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46baf25b-4f24-4202-9870-b45a7ab4e787-tls-certs\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668320    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-tmp-dir\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668367    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-model-cache\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-home\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8cnn\" (UniqueName: \"kubernetes.io/projected/46baf25b-4f24-4202-9870-b45a7ab4e787-kube-api-access-g8cnn\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668796 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668768    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-tmp-dir\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668862 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-model-cache\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.668862 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.668822    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-home\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.670490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.670463    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-dshm\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.670754 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.670736    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46baf25b-4f24-4202-9870-b45a7ab4e787-tls-certs\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.675903 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.675883    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8cnn\" (UniqueName: \"kubernetes.io/projected/46baf25b-4f24-4202-9870-b45a7ab4e787-kube-api-access-g8cnn\") pod \"e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") " pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.762030 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.761944    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:37.890641 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.890614    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"]
Jul 11 08:14:37.892843 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:14:37.892809    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46baf25b_4f24_4202_9870_b45a7ab4e787.slice/crio-9978afc42fcfaa534994fbae360658bdd6267b3f72d548c9b139bfad37b7120c WatchSource:0}: Error finding container 9978afc42fcfaa534994fbae360658bdd6267b3f72d548c9b139bfad37b7120c: Status 404 returned error can't find the container with id 9978afc42fcfaa534994fbae360658bdd6267b3f72d548c9b139bfad37b7120c
Jul 11 08:14:37.926337 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:37.926307    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr" event={"ID":"46baf25b-4f24-4202-9870-b45a7ab4e787","Type":"ContainerStarted","Data":"9978afc42fcfaa534994fbae360658bdd6267b3f72d548c9b139bfad37b7120c"}
Jul 11 08:14:38.933056 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:38.933024    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr" event={"ID":"46baf25b-4f24-4202-9870-b45a7ab4e787","Type":"ContainerStarted","Data":"6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1"}
Jul 11 08:14:38.951068 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:38.951020    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr" podStartSLOduration=1.951004598 podStartE2EDuration="1.951004598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:14:38.949641453 +0000 UTC m=+2185.153060315" watchObservedRunningTime="2026-07-11 08:14:38.951004598 +0000 UTC m=+2185.154423461"
Jul 11 08:14:46.398921 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:46.398889    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:46.497958 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:46.497923    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:14:47.762214 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:47.762172    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:47.762214 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:47.762223    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:47.774288 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:47.774264    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:14:47.983429 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:14:47.983401    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:15:11.742707 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:11.742676    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:15:13.135912 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:13.135881    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:15:21.697997 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:21.697954    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:15:26.374846 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:26.374808    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:15:31.357930 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:31.357896    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"]
Jul 11 08:15:31.456318 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:31.456284    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:15:46.378597 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.378489    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr" podUID="46baf25b-4f24-4202-9870-b45a7ab4e787" containerName="main" containerID="cri-o://6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1" gracePeriod=45
Jul 11 08:15:46.627595 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.627572    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:15:46.776675 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776588    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-model-cache\") pod \"46baf25b-4f24-4202-9870-b45a7ab4e787\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") "
Jul 11 08:15:46.776675 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776648    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-home\") pod \"46baf25b-4f24-4202-9870-b45a7ab4e787\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") "
Jul 11 08:15:46.776909 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776698    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-tmp-dir\") pod \"46baf25b-4f24-4202-9870-b45a7ab4e787\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") "
Jul 11 08:15:46.776909 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776727    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46baf25b-4f24-4202-9870-b45a7ab4e787-tls-certs\") pod \"46baf25b-4f24-4202-9870-b45a7ab4e787\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") "
Jul 11 08:15:46.776909 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776753    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-dshm\") pod \"46baf25b-4f24-4202-9870-b45a7ab4e787\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") "
Jul 11 08:15:46.776909 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776786    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8cnn\" (UniqueName: \"kubernetes.io/projected/46baf25b-4f24-4202-9870-b45a7ab4e787-kube-api-access-g8cnn\") pod \"46baf25b-4f24-4202-9870-b45a7ab4e787\" (UID: \"46baf25b-4f24-4202-9870-b45a7ab4e787\") "
Jul 11 08:15:46.777116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776918    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-model-cache" (OuterVolumeSpecName: "model-cache") pod "46baf25b-4f24-4202-9870-b45a7ab4e787" (UID: "46baf25b-4f24-4202-9870-b45a7ab4e787"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:15:46.777116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776946    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "46baf25b-4f24-4202-9870-b45a7ab4e787" (UID: "46baf25b-4f24-4202-9870-b45a7ab4e787"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:15:46.777116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.776966    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-home" (OuterVolumeSpecName: "home") pod "46baf25b-4f24-4202-9870-b45a7ab4e787" (UID: "46baf25b-4f24-4202-9870-b45a7ab4e787"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:15:46.777116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.777079    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:15:46.777116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.777093    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:15:46.777116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.777109    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:15:46.778912 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.778887    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46baf25b-4f24-4202-9870-b45a7ab4e787-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46baf25b-4f24-4202-9870-b45a7ab4e787" (UID: "46baf25b-4f24-4202-9870-b45a7ab4e787"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:15:46.779018 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.778990    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-dshm" (OuterVolumeSpecName: "dshm") pod "46baf25b-4f24-4202-9870-b45a7ab4e787" (UID: "46baf25b-4f24-4202-9870-b45a7ab4e787"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:15:46.779018 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.778984    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46baf25b-4f24-4202-9870-b45a7ab4e787-kube-api-access-g8cnn" (OuterVolumeSpecName: "kube-api-access-g8cnn") pod "46baf25b-4f24-4202-9870-b45a7ab4e787" (UID: "46baf25b-4f24-4202-9870-b45a7ab4e787"). InnerVolumeSpecName "kube-api-access-g8cnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:15:46.878057 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.878035    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46baf25b-4f24-4202-9870-b45a7ab4e787-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:15:46.878057 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.878056    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46baf25b-4f24-4202-9870-b45a7ab4e787-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:15:46.878196 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:46.878065    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8cnn\" (UniqueName: \"kubernetes.io/projected/46baf25b-4f24-4202-9870-b45a7ab4e787-kube-api-access-g8cnn\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:15:47.245151 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.245120    2572 generic.go:356] "Generic (PLEG): container finished" podID="46baf25b-4f24-4202-9870-b45a7ab4e787" containerID="6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1" exitCode=0
Jul 11 08:15:47.245318 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.245162    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"
Jul 11 08:15:47.245318 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.245203    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr" event={"ID":"46baf25b-4f24-4202-9870-b45a7ab4e787","Type":"ContainerDied","Data":"6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1"}
Jul 11 08:15:47.245318 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.245246    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr" event={"ID":"46baf25b-4f24-4202-9870-b45a7ab4e787","Type":"ContainerDied","Data":"9978afc42fcfaa534994fbae360658bdd6267b3f72d548c9b139bfad37b7120c"}
Jul 11 08:15:47.245318 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.245267    2572 scope.go:117] "RemoveContainer" containerID="6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1"
Jul 11 08:15:47.254113 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.254095    2572 scope.go:117] "RemoveContainer" containerID="6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1"
Jul 11 08:15:47.254376 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:15:47.254350    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1\": container with ID starting with 6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1 not found: ID does not exist" containerID="6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1"
Jul 11 08:15:47.254431 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.254384    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1"} err="failed to get container status \"6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1\": rpc error: code = NotFound desc = could not find container \"6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1\": container with ID starting with 6c0a990d7085aa72c7d175a1cf37b33a6d2c1d4d613f5caf91a826620594c0b1 not found: ID does not exist"
Jul 11 08:15:47.264850 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.264825    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"]
Jul 11 08:15:47.270453 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:47.270432    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-683f5ca7/e2e-lifecycle-model-683f5ca7-kserve-78965687f5-vcvfr"]
Jul 11 08:15:48.293401 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:15:48.293369    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46baf25b-4f24-4202-9870-b45a7ab4e787" path="/var/lib/kubelet/pods/46baf25b-4f24-4202-9870-b45a7ab4e787/volumes"
Jul 11 08:16:07.654744 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:07.654711    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:08.593725 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:08.593698    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:14.101543 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:14.101491    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:14.166023 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:14.165988    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:18.544479 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:18.544433    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:18.762904 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:18.762871    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:19.754259 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:19.754227    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:19.855181 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:19.855145    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:19.955820 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:19.955784    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:20.620439 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:20.620404    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:20.720275 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:20.720226    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:20.820684 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:20.820656    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:22.167689 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:22.167659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:27.980971 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:27.980933    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:28.034662 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:28.034633    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:28.757794 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:28.757761    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:34.357272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:34.357237    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:34.422006 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:34.421973    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:34.499210 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:34.499172    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:34.600408 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:34.600378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:45.054592 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:45.054550    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:16:45.392341 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:16:45.392254    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:06.334466 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:06.334389    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:11.938578 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:11.938546    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:12.040122 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:12.040094    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:12.269863 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:12.269773    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:12.568734 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:12.568701    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:12.676213 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:12.676180    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:15.319725 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:15.319665    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:15.424225 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:15.424179    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:15.588357 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:15.588267    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:15.691327 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:15.691292    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:16.122231 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:16.122198    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:16.222858 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:16.222828    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:19.816600 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:19.816559    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:20.133093 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:20.133011    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:20.428024 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:20.427940    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:20.930157 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:20.930129    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:22.979452 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:22.979421    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:23.083100 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:23.083069    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:23.319150 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:23.319083    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:23.645977 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:23.645893    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:23.863431 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:23.863395    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:23.963707 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:23.963618    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:27.455703 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:27.455670    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:36.651297 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.651262    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"]
Jul 11 08:17:36.651698 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.651631    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46baf25b-4f24-4202-9870-b45a7ab4e787" containerName="main"
Jul 11 08:17:36.651698 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.651641    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="46baf25b-4f24-4202-9870-b45a7ab4e787" containerName="main"
Jul 11 08:17:36.651772 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.651712    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="46baf25b-4f24-4202-9870-b45a7ab4e787" containerName="main"
Jul 11 08:17:36.654785 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.654766    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.657160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.657136    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8da7f7\"/\"kube-root-ca.crt\""
Jul 11 08:17:36.657160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.657154    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8da7f7\"/\"openshift-service-ca.crt\""
Jul 11 08:17:36.658339 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.658320    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8da7f7\"/\"default-dockercfg-lfkgf\""
Jul 11 08:17:36.658431 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.658348    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8da7f7\"/\"test-model-8da7f7-kserve-self-signed-certs\""
Jul 11 08:17:36.664149 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.664119    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"]
Jul 11 08:17:36.674713 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.674676    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:36.717879 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.717848    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tmp-dir\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.717879 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.717881    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-home\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.718064 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.717901    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-dshm\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.718064 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.717979    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-model-cache\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.718064 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.718009    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tls-certs\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.718064 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.718053    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p98xc\" (UniqueName: \"kubernetes.io/projected/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-kube-api-access-p98xc\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818466 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-dshm\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818662 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818481    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-model-cache\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818662 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818506    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tls-certs\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818759 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818709    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p98xc\" (UniqueName: \"kubernetes.io/projected/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-kube-api-access-p98xc\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818825 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818806    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tmp-dir\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818879 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818850    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-home\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.818985 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.818968    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-model-cache\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.819199 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.819170    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tmp-dir\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.819326 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.819222    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-home\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.821095 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.821065    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-dshm\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.821216 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.821194    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tls-certs\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.826260 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.826239    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p98xc\" (UniqueName: \"kubernetes.io/projected/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-kube-api-access-p98xc\") pod \"test-model-8da7f7-kserve-87c9b5dcf-gmj2v\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") " pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:36.949200 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.949114    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:36.964502 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:36.964471    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:37.051266 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:37.051237    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:37.096886 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:37.096856    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"]
Jul 11 08:17:37.099054 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:17:37.099024    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ca592e3_6f13_49f3_92e0_c2aa81ccd458.slice/crio-9cbc441d121699ef9cea7294cbff04825359d5723019d21e14bac7d2cac79ba6 WatchSource:0}: Error finding container 9cbc441d121699ef9cea7294cbff04825359d5723019d21e14bac7d2cac79ba6: Status 404 returned error can't find the container with id 9cbc441d121699ef9cea7294cbff04825359d5723019d21e14bac7d2cac79ba6
Jul 11 08:17:37.752848 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:37.752809    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v" event={"ID":"1ca592e3-6f13-49f3-92e0-c2aa81ccd458","Type":"ContainerStarted","Data":"8d8d949d6882ca93f883b1b4237bb37dcea58c423f3522e3c5573ae1a945b4ae"}
Jul 11 08:17:37.753312 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:37.752857    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v" event={"ID":"1ca592e3-6f13-49f3-92e0-c2aa81ccd458","Type":"ContainerStarted","Data":"9cbc441d121699ef9cea7294cbff04825359d5723019d21e14bac7d2cac79ba6"}
Jul 11 08:17:37.770274 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:37.770219    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v" podStartSLOduration=1.770204136 podStartE2EDuration="1.770204136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:17:37.769753617 +0000 UTC m=+2363.973172481" watchObservedRunningTime="2026-07-11 08:17:37.770204136 +0000 UTC m=+2363.973622998"
Jul 11 08:17:40.440228 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:40.440182    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:44.583922 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:44.583888    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:44.685361 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:44.685330    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:46.964959 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:46.964930    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:46.965390 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:46.964977    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:46.977195 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:46.977166    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:47.807835 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:47.807811    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:17:58.556693 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.556660    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"]
Jul 11 08:17:58.560086 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.560071    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.562373 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.562353    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-59f9af\"/\"default-dockercfg-9x8ps\""
Jul 11 08:17:58.562373 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.562362    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-59f9af\"/\"kube-root-ca.crt\""
Jul 11 08:17:58.563489 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.563463    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-59f9af\"/\"test-model-59f9af-kserve-self-signed-certs\""
Jul 11 08:17:58.563614 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.563502    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-59f9af\"/\"openshift-service-ca.crt\""
Jul 11 08:17:58.570973 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.570951    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"]
Jul 11 08:17:58.582426 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.582401    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:17:58.600768 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.600739    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-model-cache\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.600889 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.600777    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cl57\" (UniqueName: \"kubernetes.io/projected/e4d6744d-8592-4174-a630-0c39c391d57d-kube-api-access-6cl57\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.600889 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.600867    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-tmp-dir\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.600976 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.600904    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d6744d-8592-4174-a630-0c39c391d57d-tls-certs\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.600976 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.600954    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-home\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.601088 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.601067    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-dshm\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.701797 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.701763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-tmp-dir\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.701963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.701812    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d6744d-8592-4174-a630-0c39c391d57d-tls-certs\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.701963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.701939    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-home\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.702075 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.702006    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-dshm\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.702075 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.702028    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-model-cache\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.702075 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.702048    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cl57\" (UniqueName: \"kubernetes.io/projected/e4d6744d-8592-4174-a630-0c39c391d57d-kube-api-access-6cl57\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.702311 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.702289    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-tmp-dir\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.702376 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.702356    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-home\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.702431 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.702400    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-model-cache\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.704150 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.704127    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-dshm\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.704411 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.704390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d6744d-8592-4174-a630-0c39c391d57d-tls-certs\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.709108 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.709078    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cl57\" (UniqueName: \"kubernetes.io/projected/e4d6744d-8592-4174-a630-0c39c391d57d-kube-api-access-6cl57\") pod \"test-model-59f9af-kserve-5d966fccd5-dp885\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") " pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.870132 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.870041    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:17:58.997660 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:58.997633    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"]
Jul 11 08:17:59.000553 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:17:59.000501    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4d6744d_8592_4174_a630_0c39c391d57d.slice/crio-6bbd6ab54a4800548f33f5ff7cc7aacdd0989ed0a9e8c68cb98180e0a7475a65 WatchSource:0}: Error finding container 6bbd6ab54a4800548f33f5ff7cc7aacdd0989ed0a9e8c68cb98180e0a7475a65: Status 404 returned error can't find the container with id 6bbd6ab54a4800548f33f5ff7cc7aacdd0989ed0a9e8c68cb98180e0a7475a65
Jul 11 08:17:59.858293 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:59.858260    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885" event={"ID":"e4d6744d-8592-4174-a630-0c39c391d57d","Type":"ContainerStarted","Data":"b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651"}
Jul 11 08:17:59.858293 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:59.858299    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885" event={"ID":"e4d6744d-8592-4174-a630-0c39c391d57d","Type":"ContainerStarted","Data":"6bbd6ab54a4800548f33f5ff7cc7aacdd0989ed0a9e8c68cb98180e0a7475a65"}
Jul 11 08:17:59.876647 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:17:59.876603    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885" podStartSLOduration=1.8765838769999998 podStartE2EDuration="1.876583877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:17:59.873923368 +0000 UTC m=+2386.077342230" watchObservedRunningTime="2026-07-11 08:17:59.876583877 +0000 UTC m=+2386.080002740"
Jul 11 08:18:08.870899 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:08.870852    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:18:08.871421 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:08.870917    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:18:08.883103 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:08.883078    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:18:08.909453 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:08.909427    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:18:14.400821 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:14.400796    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:18:14.410782 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:14.410762    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:18:25.808631 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:25.808595    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"]
Jul 11 08:18:25.834006 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:25.833976    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:26.787796 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:26.787764    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"]
Jul 11 08:18:26.807078 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:26.807046    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:27.390140 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:27.390107    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:27.491177 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:27.491147    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:28.863520 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:28.863485    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:34.656051 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:34.656020    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:34.727847 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:34.727812    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:37.323016 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:37.322938    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:40.827902 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:40.827865    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v" podUID="1ca592e3-6f13-49f3-92e0-c2aa81ccd458" containerName="main" containerID="cri-o://8d8d949d6882ca93f883b1b4237bb37dcea58c423f3522e3c5573ae1a945b4ae" gracePeriod=45
Jul 11 08:18:41.056919 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.056893    2572 generic.go:356] "Generic (PLEG): container finished" podID="1ca592e3-6f13-49f3-92e0-c2aa81ccd458" containerID="8d8d949d6882ca93f883b1b4237bb37dcea58c423f3522e3c5573ae1a945b4ae" exitCode=0
Jul 11 08:18:41.057047 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.056949    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v" event={"ID":"1ca592e3-6f13-49f3-92e0-c2aa81ccd458","Type":"ContainerDied","Data":"8d8d949d6882ca93f883b1b4237bb37dcea58c423f3522e3c5573ae1a945b4ae"}
Jul 11 08:18:41.057047 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.056975    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v" event={"ID":"1ca592e3-6f13-49f3-92e0-c2aa81ccd458","Type":"ContainerDied","Data":"9cbc441d121699ef9cea7294cbff04825359d5723019d21e14bac7d2cac79ba6"}
Jul 11 08:18:41.057047 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.056987    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cbc441d121699ef9cea7294cbff04825359d5723019d21e14bac7d2cac79ba6"
Jul 11 08:18:41.066496 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.066473    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:18:41.186987 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.186912    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p98xc\" (UniqueName: \"kubernetes.io/projected/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-kube-api-access-p98xc\") pod \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") "
Jul 11 08:18:41.186987 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.186966    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-model-cache\") pod \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") "
Jul 11 08:18:41.187216 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187036    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tls-certs\") pod \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") "
Jul 11 08:18:41.187216 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187056    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-dshm\") pod \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") "
Jul 11 08:18:41.187216 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187112    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-home\") pod \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") "
Jul 11 08:18:41.187216 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187140    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tmp-dir\") pod \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\" (UID: \"1ca592e3-6f13-49f3-92e0-c2aa81ccd458\") "
Jul 11 08:18:41.187487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187253    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ca592e3-6f13-49f3-92e0-c2aa81ccd458" (UID: "1ca592e3-6f13-49f3-92e0-c2aa81ccd458"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:41.187487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187376    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-home" (OuterVolumeSpecName: "home") pod "1ca592e3-6f13-49f3-92e0-c2aa81ccd458" (UID: "1ca592e3-6f13-49f3-92e0-c2aa81ccd458"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:41.187487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187408    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ca592e3-6f13-49f3-92e0-c2aa81ccd458" (UID: "1ca592e3-6f13-49f3-92e0-c2aa81ccd458"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:41.187487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187455    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:41.187487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.187476    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:41.189259 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.189228    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ca592e3-6f13-49f3-92e0-c2aa81ccd458" (UID: "1ca592e3-6f13-49f3-92e0-c2aa81ccd458"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:18:41.189259 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.189231    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-dshm" (OuterVolumeSpecName: "dshm") pod "1ca592e3-6f13-49f3-92e0-c2aa81ccd458" (UID: "1ca592e3-6f13-49f3-92e0-c2aa81ccd458"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:41.189428 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.189263    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-kube-api-access-p98xc" (OuterVolumeSpecName: "kube-api-access-p98xc") pod "1ca592e3-6f13-49f3-92e0-c2aa81ccd458" (UID: "1ca592e3-6f13-49f3-92e0-c2aa81ccd458"). InnerVolumeSpecName "kube-api-access-p98xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:18:41.288424 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.288394    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:41.288424 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.288418    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:41.288424 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.288425    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:41.288635 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.288434    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p98xc\" (UniqueName: \"kubernetes.io/projected/1ca592e3-6f13-49f3-92e0-c2aa81ccd458-kube-api-access-p98xc\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:41.808841 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:41.808789    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885" podUID="e4d6744d-8592-4174-a630-0c39c391d57d" containerName="main" containerID="cri-o://b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651" gracePeriod=45
Jul 11 08:18:42.005418 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.005379    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:42.056438 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.056412    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:18:42.063315 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.063256    2572 generic.go:356] "Generic (PLEG): container finished" podID="e4d6744d-8592-4174-a630-0c39c391d57d" containerID="b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651" exitCode=0
Jul 11 08:18:42.063315 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.063290    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"
Jul 11 08:18:42.063508 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.063330    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885" event={"ID":"e4d6744d-8592-4174-a630-0c39c391d57d","Type":"ContainerDied","Data":"b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651"}
Jul 11 08:18:42.063508 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.063367    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885" event={"ID":"e4d6744d-8592-4174-a630-0c39c391d57d","Type":"ContainerDied","Data":"6bbd6ab54a4800548f33f5ff7cc7aacdd0989ed0a9e8c68cb98180e0a7475a65"}
Jul 11 08:18:42.063508 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.063384    2572 scope.go:117] "RemoveContainer" containerID="b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651"
Jul 11 08:18:42.063508 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.063389    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"
Jul 11 08:18:42.071586 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.071568    2572 scope.go:117] "RemoveContainer" containerID="b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651"
Jul 11 08:18:42.071857 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:18:42.071839    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651\": container with ID starting with b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651 not found: ID does not exist" containerID="b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651"
Jul 11 08:18:42.071916 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.071866    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651"} err="failed to get container status \"b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651\": rpc error: code = NotFound desc = could not find container \"b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651\": container with ID starting with b16a5a51090d59ae39a5ecb0bab9bc99cf20325c14c4bcc459318047d2795651 not found: ID does not exist"
Jul 11 08:18:42.084449 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.084421    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"]
Jul 11 08:18:42.088182 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.088159    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8da7f7/test-model-8da7f7-kserve-87c9b5dcf-gmj2v"]
Jul 11 08:18:42.197016 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.196966    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d6744d-8592-4174-a630-0c39c391d57d-tls-certs\") pod \"e4d6744d-8592-4174-a630-0c39c391d57d\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") "
Jul 11 08:18:42.197016 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197022    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cl57\" (UniqueName: \"kubernetes.io/projected/e4d6744d-8592-4174-a630-0c39c391d57d-kube-api-access-6cl57\") pod \"e4d6744d-8592-4174-a630-0c39c391d57d\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") "
Jul 11 08:18:42.197272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197051    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-home\") pod \"e4d6744d-8592-4174-a630-0c39c391d57d\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") "
Jul 11 08:18:42.197272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197065    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-dshm\") pod \"e4d6744d-8592-4174-a630-0c39c391d57d\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") "
Jul 11 08:18:42.197272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197152    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-model-cache\") pod \"e4d6744d-8592-4174-a630-0c39c391d57d\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") "
Jul 11 08:18:42.197272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197169    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-tmp-dir\") pod \"e4d6744d-8592-4174-a630-0c39c391d57d\" (UID: \"e4d6744d-8592-4174-a630-0c39c391d57d\") "
Jul 11 08:18:42.197481 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197372    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-home" (OuterVolumeSpecName: "home") pod "e4d6744d-8592-4174-a630-0c39c391d57d" (UID: "e4d6744d-8592-4174-a630-0c39c391d57d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:42.197481 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197419    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4d6744d-8592-4174-a630-0c39c391d57d" (UID: "e4d6744d-8592-4174-a630-0c39c391d57d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:42.197615 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.197507    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4d6744d-8592-4174-a630-0c39c391d57d" (UID: "e4d6744d-8592-4174-a630-0c39c391d57d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:42.199356 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.199329    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-dshm" (OuterVolumeSpecName: "dshm") pod "e4d6744d-8592-4174-a630-0c39c391d57d" (UID: "e4d6744d-8592-4174-a630-0c39c391d57d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:18:42.199356 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.199343    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4d6744d-8592-4174-a630-0c39c391d57d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4d6744d-8592-4174-a630-0c39c391d57d" (UID: "e4d6744d-8592-4174-a630-0c39c391d57d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:18:42.199500 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.199339    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d6744d-8592-4174-a630-0c39c391d57d-kube-api-access-6cl57" (OuterVolumeSpecName: "kube-api-access-6cl57") pod "e4d6744d-8592-4174-a630-0c39c391d57d" (UID: "e4d6744d-8592-4174-a630-0c39c391d57d"). InnerVolumeSpecName "kube-api-access-6cl57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:18:42.293826 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.293788    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ca592e3-6f13-49f3-92e0-c2aa81ccd458" path="/var/lib/kubelet/pods/1ca592e3-6f13-49f3-92e0-c2aa81ccd458/volumes"
Jul 11 08:18:42.297759 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.297736    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d6744d-8592-4174-a630-0c39c391d57d-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:42.297759 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.297756    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cl57\" (UniqueName: \"kubernetes.io/projected/e4d6744d-8592-4174-a630-0c39c391d57d-kube-api-access-6cl57\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:42.297956 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.297766    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:42.297956 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.297776    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:42.297956 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.297784    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:42.297956 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.297792    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d6744d-8592-4174-a630-0c39c391d57d-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:42.378601 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.378515    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"]
Jul 11 08:18:42.381944 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:42.381923    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-59f9af/test-model-59f9af-kserve-5d966fccd5-dp885"]
Jul 11 08:18:44.294017 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:44.293987    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d6744d-8592-4174-a630-0c39c391d57d" path="/var/lib/kubelet/pods/e4d6744d-8592-4174-a630-0c39c391d57d/volumes"
Jul 11 08:18:47.690608 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:47.690578    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:47.789008 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:47.788972    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:52.683711 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:52.683678    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:52.901452 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:52.901419    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:53.026816 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.026731    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"]
Jul 11 08:18:53.027158 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.027140    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca592e3-6f13-49f3-92e0-c2aa81ccd458" containerName="main"
Jul 11 08:18:53.027250 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.027161    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca592e3-6f13-49f3-92e0-c2aa81ccd458" containerName="main"
Jul 11 08:18:53.027250 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.027179    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d6744d-8592-4174-a630-0c39c391d57d" containerName="main"
Jul 11 08:18:53.027250 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.027187    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d6744d-8592-4174-a630-0c39c391d57d" containerName="main"
Jul 11 08:18:53.027396 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.027305    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d6744d-8592-4174-a630-0c39c391d57d" containerName="main"
Jul 11 08:18:53.027396 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.027321    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca592e3-6f13-49f3-92e0-c2aa81ccd458" containerName="main"
Jul 11 08:18:53.030750 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.030730    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:18:53.033104 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.033075    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 11 08:18:53.033216 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.033110    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\""
Jul 11 08:18:53.037565 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.037542    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"]
Jul 11 08:18:53.196831 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.196796    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7511eab5-d8b2-4314-a6a7-198038cd0058-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52\" (UID: \"7511eab5-d8b2-4314-a6a7-198038cd0058\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:18:53.297810 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.297734    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7511eab5-d8b2-4314-a6a7-198038cd0058-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52\" (UID: \"7511eab5-d8b2-4314-a6a7-198038cd0058\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:18:53.298309 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.298289    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7511eab5-d8b2-4314-a6a7-198038cd0058-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52\" (UID: \"7511eab5-d8b2-4314-a6a7-198038cd0058\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:18:53.342377 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.342346    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:18:53.469770 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.469746    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"]
Jul 11 08:18:53.472143 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:18:53.472098    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7511eab5_d8b2_4314_a6a7_198038cd0058.slice/crio-d528b4b9c85f71af3f664c7b5bc4fca775a35d2dc734ada3aabf657a9aa219d2 WatchSource:0}: Error finding container d528b4b9c85f71af3f664c7b5bc4fca775a35d2dc734ada3aabf657a9aa219d2: Status 404 returned error can't find the container with id d528b4b9c85f71af3f664c7b5bc4fca775a35d2dc734ada3aabf657a9aa219d2
Jul 11 08:18:53.473906 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.473885    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 08:18:53.715685 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.715655    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:53.815851 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:53.815820    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:18:54.124708 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:54.124680    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52" event={"ID":"7511eab5-d8b2-4314-a6a7-198038cd0058","Type":"ContainerStarted","Data":"d528b4b9c85f71af3f664c7b5bc4fca775a35d2dc734ada3aabf657a9aa219d2"}
Jul 11 08:18:56.136390 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:56.136365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52" event={"ID":"7511eab5-d8b2-4314-a6a7-198038cd0058","Type":"ContainerStarted","Data":"8e2090b3835299368b9d6a5c07ca1c57afa9f5882c5dfdaf67f1682c71540248"}
Jul 11 08:18:56.151587 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:56.151541    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52" podStartSLOduration=0.685429596 podStartE2EDuration="3.151512087s" totalImagesPullingTime="2.466082491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:18:56.150271509 +0000 UTC m=+2442.353690373" watchObservedRunningTime="2026-07-11 08:18:56.151512087 +0000 UTC m=+2442.354930949"
Jul 11 08:18:57.142716 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:57.142682    2572 generic.go:356] "Generic (PLEG): container finished" podID="7511eab5-d8b2-4314-a6a7-198038cd0058" containerID="8e2090b3835299368b9d6a5c07ca1c57afa9f5882c5dfdaf67f1682c71540248" exitCode=0
Jul 11 08:18:57.143052 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:57.142760    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52" event={"ID":"7511eab5-d8b2-4314-a6a7-198038cd0058","Type":"ContainerDied","Data":"8e2090b3835299368b9d6a5c07ca1c57afa9f5882c5dfdaf67f1682c71540248"}
Jul 11 08:18:58.276201 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:58.276179    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:18:58.446877 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:58.446793    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7511eab5-d8b2-4314-a6a7-198038cd0058-maas-api-service-ca\") pod \"7511eab5-d8b2-4314-a6a7-198038cd0058\" (UID: \"7511eab5-d8b2-4314-a6a7-198038cd0058\") "
Jul 11 08:18:58.447180 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:58.447148    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7511eab5-d8b2-4314-a6a7-198038cd0058-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7511eab5-d8b2-4314-a6a7-198038cd0058" (UID: "7511eab5-d8b2-4314-a6a7-198038cd0058"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 08:18:58.547794 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:58.547768    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7511eab5-d8b2-4314-a6a7-198038cd0058-maas-api-service-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:18:59.153050 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:59.153018    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52" event={"ID":"7511eab5-d8b2-4314-a6a7-198038cd0058","Type":"ContainerDied","Data":"d528b4b9c85f71af3f664c7b5bc4fca775a35d2dc734ada3aabf657a9aa219d2"}
Jul 11 08:18:59.153221 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:59.153065    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d528b4b9c85f71af3f664c7b5bc4fca775a35d2dc734ada3aabf657a9aa219d2"
Jul 11 08:18:59.153221 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:18:59.153028    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52"
Jul 11 08:19:03.452278 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:03.452247    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:06.999457 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:06.999423    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:08.113357 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:08.113322    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:13.887472 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:13.887437    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:13.987434 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:13.987400    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:18.809122 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:18.809091    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:19.041173 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:19.041125    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:19.380962 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:19.380935    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:19.698819 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:19.698732    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:20.133598 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:20.133564    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:23.011272 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:23.011237    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:23.164187 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:23.164158    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:23.261590 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:23.261480    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:23.428350 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:23.428315    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:23.703771 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:23.703740    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:23.803544 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:23.803489    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:27.513068 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:27.513034    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:28.001701 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:27.999399    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:28.314333 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:28.314301    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:28.537751 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:28.537721    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:30.813246 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:30.813212    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:30.907733 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:30.907695    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:30.978796 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:30.978768    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:31.088794 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:31.088714    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:31.515625 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:31.515545    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:31.616551 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:31.613709    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:35.317096 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:35.317063    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:43.414960 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.414926    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"]
Jul 11 08:19:43.415341 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.415285    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7511eab5-d8b2-4314-a6a7-198038cd0058" containerName="revoke-keys"
Jul 11 08:19:43.415341 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.415296    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7511eab5-d8b2-4314-a6a7-198038cd0058" containerName="revoke-keys"
Jul 11 08:19:43.415411 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.415368    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7511eab5-d8b2-4314-a6a7-198038cd0058" containerName="revoke-keys"
Jul 11 08:19:43.418426 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.418407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:43.420658 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.420627    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 11 08:19:43.420776 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.420682    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\""
Jul 11 08:19:43.425009 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.424987    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"]
Jul 11 08:19:43.545927 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.545888    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd85e9e8-9b75-4a36-ae34-542e11271559-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-393a88-ptgkx\" (UID: \"fd85e9e8-9b75-4a36-ae34-542e11271559\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:43.647354 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.647317    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd85e9e8-9b75-4a36-ae34-542e11271559-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-393a88-ptgkx\" (UID: \"fd85e9e8-9b75-4a36-ae34-542e11271559\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:43.647945 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.647927    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd85e9e8-9b75-4a36-ae34-542e11271559-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-393a88-ptgkx\" (UID: \"fd85e9e8-9b75-4a36-ae34-542e11271559\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:43.729449 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.729370    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:43.860321 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:43.860292    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"]
Jul 11 08:19:43.864005 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:19:43.863973    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd85e9e8_9b75_4a36_ae34_542e11271559.slice/crio-b45c17ea6753be6c3184c1e9db04871c0fe01a7dcefa9096bbcca13a79727f92 WatchSource:0}: Error finding container b45c17ea6753be6c3184c1e9db04871c0fe01a7dcefa9096bbcca13a79727f92: Status 404 returned error can't find the container with id b45c17ea6753be6c3184c1e9db04871c0fe01a7dcefa9096bbcca13a79727f92
Jul 11 08:19:44.069213 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:44.069184    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:44.168097 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:44.168067    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:44.357946 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:44.357852    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx" event={"ID":"fd85e9e8-9b75-4a36-ae34-542e11271559","Type":"ContainerStarted","Data":"a86b3ed5226ac3af2d850a1f182f19e38aa6017e3834389834ef8ddb47a54631"}
Jul 11 08:19:44.357946 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:44.357899    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx" event={"ID":"fd85e9e8-9b75-4a36-ae34-542e11271559","Type":"ContainerStarted","Data":"b45c17ea6753be6c3184c1e9db04871c0fe01a7dcefa9096bbcca13a79727f92"}
Jul 11 08:19:44.372662 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:44.372606    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx" podStartSLOduration=1.37259134 podStartE2EDuration="1.37259134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:19:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:19:44.370454642 +0000 UTC m=+2490.573873504" watchObservedRunningTime="2026-07-11 08:19:44.37259134 +0000 UTC m=+2490.576010202"
Jul 11 08:19:45.363623 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:45.363593    2572 generic.go:356] "Generic (PLEG): container finished" podID="fd85e9e8-9b75-4a36-ae34-542e11271559" containerID="a86b3ed5226ac3af2d850a1f182f19e38aa6017e3834389834ef8ddb47a54631" exitCode=0
Jul 11 08:19:45.364067 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:45.363655    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx" event={"ID":"fd85e9e8-9b75-4a36-ae34-542e11271559","Type":"ContainerDied","Data":"a86b3ed5226ac3af2d850a1f182f19e38aa6017e3834389834ef8ddb47a54631"}
Jul 11 08:19:46.502851 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:46.502830    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:46.678905 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:46.678825    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd85e9e8-9b75-4a36-ae34-542e11271559-maas-api-service-ca\") pod \"fd85e9e8-9b75-4a36-ae34-542e11271559\" (UID: \"fd85e9e8-9b75-4a36-ae34-542e11271559\") "
Jul 11 08:19:46.679191 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:46.679161    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd85e9e8-9b75-4a36-ae34-542e11271559-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd85e9e8-9b75-4a36-ae34-542e11271559" (UID: "fd85e9e8-9b75-4a36-ae34-542e11271559"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 08:19:46.780280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:46.780248    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd85e9e8-9b75-4a36-ae34-542e11271559-maas-api-service-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:19:47.254559 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:47.254515    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:47.354100 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:47.354062    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:47.376346 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:47.376319    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx"
Jul 11 08:19:47.376465 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:47.376318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-393a88-ptgkx" event={"ID":"fd85e9e8-9b75-4a36-ae34-542e11271559","Type":"ContainerDied","Data":"b45c17ea6753be6c3184c1e9db04871c0fe01a7dcefa9096bbcca13a79727f92"}
Jul 11 08:19:47.376465 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:47.376440    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b45c17ea6753be6c3184c1e9db04871c0fe01a7dcefa9096bbcca13a79727f92"
Jul 11 08:19:52.253170 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:52.253135    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:52.353080 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:52.353035    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:53.967885 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:53.967849    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:54.067923 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:54.067891    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:54.202261 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:54.202230    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:19:58.499151 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:19:58.499109    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:04.217269 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:04.217225    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:04.399170 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:04.399138    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:09.183361 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.183286    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:09.391048 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.391015    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:09.504022 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.503936    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"]
Jul 11 08:20:09.504318 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.504306    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd85e9e8-9b75-4a36-ae34-542e11271559" containerName="revoke-keys"
Jul 11 08:20:09.504355 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.504319    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd85e9e8-9b75-4a36-ae34-542e11271559" containerName="revoke-keys"
Jul 11 08:20:09.504388 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.504379    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd85e9e8-9b75-4a36-ae34-542e11271559" containerName="revoke-keys"
Jul 11 08:20:09.507458 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.507443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:09.509778 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.509757    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\""
Jul 11 08:20:09.509885 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.509792    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 11 08:20:09.514858 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.514840    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"]
Jul 11 08:20:09.583918 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.583888    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d07a0011-68a9-4f7a-9818-daa3b78bcc30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr\" (UID: \"d07a0011-68a9-4f7a-9818-daa3b78bcc30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:09.685280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.685245    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d07a0011-68a9-4f7a-9818-daa3b78bcc30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr\" (UID: \"d07a0011-68a9-4f7a-9818-daa3b78bcc30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:09.685879 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.685861    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d07a0011-68a9-4f7a-9818-daa3b78bcc30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr\" (UID: \"d07a0011-68a9-4f7a-9818-daa3b78bcc30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:09.819034 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.818988    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:09.953986 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:09.953920    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"]
Jul 11 08:20:09.956583 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:20:09.956558    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07a0011_68a9_4f7a_9818_daa3b78bcc30.slice/crio-691ed6e7747b4c865acbe27ba0ebeaa26d67a119f35a755365345a13c4be5175 WatchSource:0}: Error finding container 691ed6e7747b4c865acbe27ba0ebeaa26d67a119f35a755365345a13c4be5175: Status 404 returned error can't find the container with id 691ed6e7747b4c865acbe27ba0ebeaa26d67a119f35a755365345a13c4be5175
Jul 11 08:20:10.092564 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:10.092447    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:10.195633 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:10.195600    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:10.486688 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:10.486600    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr" event={"ID":"d07a0011-68a9-4f7a-9818-daa3b78bcc30","Type":"ContainerStarted","Data":"7c348329c2bdb89c59862c833352817c0ea1eadd7b44b23c735927b3a1ceee61"}
Jul 11 08:20:10.486688 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:10.486651    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr" event={"ID":"d07a0011-68a9-4f7a-9818-daa3b78bcc30","Type":"ContainerStarted","Data":"691ed6e7747b4c865acbe27ba0ebeaa26d67a119f35a755365345a13c4be5175"}
Jul 11 08:20:10.501310 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:10.501264    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr" podStartSLOduration=1.5012476750000001 podStartE2EDuration="1.501247675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:20:10.499909832 +0000 UTC m=+2516.703328695" watchObservedRunningTime="2026-07-11 08:20:10.501247675 +0000 UTC m=+2516.704666538"
Jul 11 08:20:11.493172 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:11.493141    2572 generic.go:356] "Generic (PLEG): container finished" podID="d07a0011-68a9-4f7a-9818-daa3b78bcc30" containerID="7c348329c2bdb89c59862c833352817c0ea1eadd7b44b23c735927b3a1ceee61" exitCode=0
Jul 11 08:20:11.493555 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:11.493198    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr" event={"ID":"d07a0011-68a9-4f7a-9818-daa3b78bcc30","Type":"ContainerDied","Data":"7c348329c2bdb89c59862c833352817c0ea1eadd7b44b23c735927b3a1ceee61"}
Jul 11 08:20:12.624938 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:12.624915    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:12.713971 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:12.713933    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d07a0011-68a9-4f7a-9818-daa3b78bcc30-maas-api-service-ca\") pod \"d07a0011-68a9-4f7a-9818-daa3b78bcc30\" (UID: \"d07a0011-68a9-4f7a-9818-daa3b78bcc30\") "
Jul 11 08:20:12.714269 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:12.714246    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d07a0011-68a9-4f7a-9818-daa3b78bcc30-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d07a0011-68a9-4f7a-9818-daa3b78bcc30" (UID: "d07a0011-68a9-4f7a-9818-daa3b78bcc30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 08:20:12.815285 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:12.815260    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d07a0011-68a9-4f7a-9818-daa3b78bcc30-maas-api-service-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:20:13.503135 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:13.503105    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr" event={"ID":"d07a0011-68a9-4f7a-9818-daa3b78bcc30","Type":"ContainerDied","Data":"691ed6e7747b4c865acbe27ba0ebeaa26d67a119f35a755365345a13c4be5175"}
Jul 11 08:20:13.503135 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:13.503128    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr"
Jul 11 08:20:13.503340 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:13.503154    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="691ed6e7747b4c865acbe27ba0ebeaa26d67a119f35a755365345a13c4be5175"
Jul 11 08:20:20.078549 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:20.078489    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:24.590360 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:24.590330    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:30.146783 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:30.146748    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:30.205903 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:30.205875    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:35.271007 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:35.270974    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:35.498712 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:35.498674    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:35.844728 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:35.844700    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:36.147638 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:36.147554    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:36.366208 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:36.366176    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:39.291786 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:39.291747    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:39.368763 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:39.368732    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:39.433558 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:39.433512    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:39.769283 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:39.769251    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:39.871208 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:39.871177    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:39.971178 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:39.971148    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:43.775492 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:43.775449    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:44.099240 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:44.099203    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:44.423171 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:44.423078    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:44.819620 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:44.819591    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:47.195836 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:47.195793    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:47.301706 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:47.301674    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:47.395867 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:47.395833    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:47.870422 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:47.870379    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:47.962686 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:47.962658    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:48.062994 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:48.062966    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:20:51.676112 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:20:51.676076    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:00.667156 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:00.667119    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:00.765807 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:00.765772    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:03.240017 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.239986    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"]
Jul 11 08:21:03.240521 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.240502    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07a0011-68a9-4f7a-9818-daa3b78bcc30" containerName="revoke-keys"
Jul 11 08:21:03.240521 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.240538    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07a0011-68a9-4f7a-9818-daa3b78bcc30" containerName="revoke-keys"
Jul 11 08:21:03.240739 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.240684    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07a0011-68a9-4f7a-9818-daa3b78bcc30" containerName="revoke-keys"
Jul 11 08:21:03.243755 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.243735    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.246404 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.246378    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"default-dockercfg-chfh6\""
Jul 11 08:21:03.246505 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.246432    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"kube-root-ca.crt\""
Jul 11 08:21:03.246505 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.246445    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"openshift-service-ca.crt\""
Jul 11 08:21:03.247441 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.247423    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"auth-test-model-f178a6-kserve-self-signed-certs\""
Jul 11 08:21:03.252705 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.252682    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"]
Jul 11 08:21:03.274065 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.274031    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:03.369101 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.369066    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tmp-dir\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.369280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.369110    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tls-certs\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.369280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.369142    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-model-cache\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.369280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.369188    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-home\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.369280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.369222    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl2ls\" (UniqueName: \"kubernetes.io/projected/da84d4bd-9232-43b7-a8e8-fd2305d5135e-kube-api-access-bl2ls\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.369417 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.369293    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-dshm\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.470627 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.470593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-dshm\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.470825 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.470655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tmp-dir\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.470825 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.470690    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tls-certs\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.470825 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.470728    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-model-cache\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.470825 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.470759    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-home\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.470825 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.470808    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl2ls\" (UniqueName: \"kubernetes.io/projected/da84d4bd-9232-43b7-a8e8-fd2305d5135e-kube-api-access-bl2ls\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.471081 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.471058    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tmp-dir\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.471129 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.471095    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-home\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.471129 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.471119    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-model-cache\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.472755 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.472730    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-dshm\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.473140 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.473117    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tls-certs\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.478468 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.478446    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl2ls\" (UniqueName: \"kubernetes.io/projected/da84d4bd-9232-43b7-a8e8-fd2305d5135e-kube-api-access-bl2ls\") pod \"auth-test-model-f178a6-kserve-76fb9794cf-tvrjj\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") " pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.554260 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.554233    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:03.685426 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.685399    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"]
Jul 11 08:21:03.687690 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:21:03.687663    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda84d4bd_9232_43b7_a8e8_fd2305d5135e.slice/crio-315ea2e2513821dab83508d49eebb34a0b9704096654d8e8d1a992a5619782f0 WatchSource:0}: Error finding container 315ea2e2513821dab83508d49eebb34a0b9704096654d8e8d1a992a5619782f0: Status 404 returned error can't find the container with id 315ea2e2513821dab83508d49eebb34a0b9704096654d8e8d1a992a5619782f0
Jul 11 08:21:03.743062 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:03.743037    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" event={"ID":"da84d4bd-9232-43b7-a8e8-fd2305d5135e","Type":"ContainerStarted","Data":"315ea2e2513821dab83508d49eebb34a0b9704096654d8e8d1a992a5619782f0"}
Jul 11 08:21:04.748783 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:04.748746    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" event={"ID":"da84d4bd-9232-43b7-a8e8-fd2305d5135e","Type":"ContainerStarted","Data":"cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a"}
Jul 11 08:21:04.763985 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:04.763918    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" podStartSLOduration=1.7638962249999999 podStartE2EDuration="1.763896225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:21:04.763227849 +0000 UTC m=+2570.966646730" watchObservedRunningTime="2026-07-11 08:21:04.763896225 +0000 UTC m=+2570.967315090"
Jul 11 08:21:09.627227 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:09.627195    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:09.726089 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:09.726057    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:13.555263 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:13.555234    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:13.555659 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:13.555276    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:13.569612 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:13.569586    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:13.801757 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:13.801726    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:21:25.454109 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.454075    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"]
Jul 11 08:21:25.457931 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.457910    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.461738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.461634    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"default-dockercfg-kmvlk\""
Jul 11 08:21:25.461738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.461653    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"kube-root-ca.crt\""
Jul 11 08:21:25.461943 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.461813    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"openshift-service-ca.crt\""
Jul 11 08:21:25.464646 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.462304    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"auth-test-model-12fff0-kserve-self-signed-certs\""
Jul 11 08:21:25.471629 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.471591    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"]
Jul 11 08:21:25.485121 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.485097    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:25.570512 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.570475    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-dshm\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.570718 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.570595    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn9gt\" (UniqueName: \"kubernetes.io/projected/2773fdef-e591-4142-a47a-b7ee30a26ea7-kube-api-access-bn9gt\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.570718 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.570638    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-tmp-dir\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.570718 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.570657    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2773fdef-e591-4142-a47a-b7ee30a26ea7-tls-certs\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.570718 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.570709    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-home\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.570891 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.570731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-model-cache\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.671763 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.671729    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn9gt\" (UniqueName: \"kubernetes.io/projected/2773fdef-e591-4142-a47a-b7ee30a26ea7-kube-api-access-bn9gt\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.671963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.671771    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-tmp-dir\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.671963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.671787    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2773fdef-e591-4142-a47a-b7ee30a26ea7-tls-certs\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.671963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.671818    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-home\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.671963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.671843    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-model-cache\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.671963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.671882    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-dshm\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.672189 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.672165    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-tmp-dir\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.672228 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.672205    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-home\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.672282 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.672264    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-model-cache\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.673996 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.673975    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-dshm\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.674286 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.674269    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2773fdef-e591-4142-a47a-b7ee30a26ea7-tls-certs\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.679099 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.679082    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn9gt\" (UniqueName: \"kubernetes.io/projected/2773fdef-e591-4142-a47a-b7ee30a26ea7-kube-api-access-bn9gt\") pod \"auth-test-model-12fff0-kserve-6c96685bd7-8rcj8\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") " pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.775963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.775881    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:25.902661 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:25.902625    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"]
Jul 11 08:21:25.905879 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:21:25.905855    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2773fdef_e591_4142_a47a_b7ee30a26ea7.slice/crio-f8871eb61e2066b9a24fce50fe3a31242ff413e3c682fbf2a0d81a8fc55cb15f WatchSource:0}: Error finding container f8871eb61e2066b9a24fce50fe3a31242ff413e3c682fbf2a0d81a8fc55cb15f: Status 404 returned error can't find the container with id f8871eb61e2066b9a24fce50fe3a31242ff413e3c682fbf2a0d81a8fc55cb15f
Jul 11 08:21:26.855506 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:26.855478    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8" event={"ID":"2773fdef-e591-4142-a47a-b7ee30a26ea7","Type":"ContainerStarted","Data":"d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e"}
Jul 11 08:21:26.855905 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:26.855515    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8" event={"ID":"2773fdef-e591-4142-a47a-b7ee30a26ea7","Type":"ContainerStarted","Data":"f8871eb61e2066b9a24fce50fe3a31242ff413e3c682fbf2a0d81a8fc55cb15f"}
Jul 11 08:21:26.872918 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:26.872873    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8" podStartSLOduration=1.872859675 podStartE2EDuration="1.872859675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:21:26.871036296 +0000 UTC m=+2593.074455159" watchObservedRunningTime="2026-07-11 08:21:26.872859675 +0000 UTC m=+2593.076278537"
Jul 11 08:21:30.409586 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:30.409554    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:21:35.776674 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:35.776600    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:35.776674 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:35.776643    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:35.789576 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:35.789551    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:21:35.909091 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:21:35.909067    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:22:07.881668 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:22:07.881633    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:23:14.430964 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:14.430891    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:23:14.439909 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:14.439888    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8lm75_a60fee7f-75a5-4e89-93f0-e8f5e571475a/ovn-acl-logging/0.log"
Jul 11 08:23:18.538232 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:18.538199    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"]
Jul 11 08:23:18.551915 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:18.551887    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:23:19.359508 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:19.359476    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"]
Jul 11 08:23:19.375869 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:19.375839    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:23:27.864229 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.864192    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"]
Jul 11 08:23:27.867742 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.867722    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:27.869873 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.869852    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"sub-test-model-f178a6-kserve-self-signed-certs\""
Jul 11 08:23:27.877384 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.877357    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"]
Jul 11 08:23:27.889635 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.889603    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:23:27.958094 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.958060    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-model-cache\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:27.958094 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.958094    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8x9x\" (UniqueName: \"kubernetes.io/projected/79965395-e2f0-48ee-bcea-0726a0b97346-kube-api-access-t8x9x\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:27.958304 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.958134    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79965395-e2f0-48ee-bcea-0726a0b97346-tls-certs\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:27.958304 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.958160    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-home\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:27.958304 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.958173    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-dshm\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:27.958304 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:27.958280    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-tmp-dir\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059613 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059580    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-model-cache\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059613 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059621    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8x9x\" (UniqueName: \"kubernetes.io/projected/79965395-e2f0-48ee-bcea-0726a0b97346-kube-api-access-t8x9x\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059841 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059800    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79965395-e2f0-48ee-bcea-0726a0b97346-tls-certs\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059841 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059824    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-home\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059841 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059837    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-dshm\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059995 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059885    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-tmp-dir\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.059995 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.059981    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-model-cache\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.060148 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.060123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-home\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.060281 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.060204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-tmp-dir\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.061962 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.061943    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-dshm\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.062287 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.062270    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79965395-e2f0-48ee-bcea-0726a0b97346-tls-certs\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.067513 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.067491    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8x9x\" (UniqueName: \"kubernetes.io/projected/79965395-e2f0-48ee-bcea-0726a0b97346-kube-api-access-t8x9x\") pod \"sub-test-model-f178a6-kserve-77bcf89cb5-62ck4\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") " pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.178709 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.178624    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:28.308860 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.308836    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"]
Jul 11 08:23:28.311287 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:23:28.311248    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79965395_e2f0_48ee_bcea_0726a0b97346.slice/crio-ce921b94e4547733ef54d6c8cc03a8961f2f9f025f5d45589d6c4ea800c584fe WatchSource:0}: Error finding container ce921b94e4547733ef54d6c8cc03a8961f2f9f025f5d45589d6c4ea800c584fe: Status 404 returned error can't find the container with id ce921b94e4547733ef54d6c8cc03a8961f2f9f025f5d45589d6c4ea800c584fe
Jul 11 08:23:28.443643 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.443555    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4" event={"ID":"79965395-e2f0-48ee-bcea-0726a0b97346","Type":"ContainerStarted","Data":"728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1"}
Jul 11 08:23:28.443643 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.443602    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4" event={"ID":"79965395-e2f0-48ee-bcea-0726a0b97346","Type":"ContainerStarted","Data":"ce921b94e4547733ef54d6c8cc03a8961f2f9f025f5d45589d6c4ea800c584fe"}
Jul 11 08:23:28.459249 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:28.459201    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4" podStartSLOduration=1.459187002 podStartE2EDuration="1.459187002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:23:28.458179804 +0000 UTC m=+2714.661598667" watchObservedRunningTime="2026-07-11 08:23:28.459187002 +0000 UTC m=+2714.662605865"
Jul 11 08:23:33.557982 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.557942    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" podUID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" containerName="main" containerID="cri-o://cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a" gracePeriod=45
Jul 11 08:23:33.803896 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.803870    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:23:33.913120 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913039    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-dshm\") pod \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") "
Jul 11 08:23:33.913120 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913095    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl2ls\" (UniqueName: \"kubernetes.io/projected/da84d4bd-9232-43b7-a8e8-fd2305d5135e-kube-api-access-bl2ls\") pod \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") "
Jul 11 08:23:33.913313 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913166    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-home\") pod \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") "
Jul 11 08:23:33.913313 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913191    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tmp-dir\") pod \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") "
Jul 11 08:23:33.913412 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913332    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tls-certs\") pod \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") "
Jul 11 08:23:33.913412 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913382    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-model-cache\") pod \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\" (UID: \"da84d4bd-9232-43b7-a8e8-fd2305d5135e\") "
Jul 11 08:23:33.913520 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913407    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-home" (OuterVolumeSpecName: "home") pod "da84d4bd-9232-43b7-a8e8-fd2305d5135e" (UID: "da84d4bd-9232-43b7-a8e8-fd2305d5135e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:33.913520 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913421    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "da84d4bd-9232-43b7-a8e8-fd2305d5135e" (UID: "da84d4bd-9232-43b7-a8e8-fd2305d5135e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:33.913680 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913659    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-model-cache" (OuterVolumeSpecName: "model-cache") pod "da84d4bd-9232-43b7-a8e8-fd2305d5135e" (UID: "da84d4bd-9232-43b7-a8e8-fd2305d5135e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:33.913809 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913791    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:33.913875 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913814    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:33.913875 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.913827    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:33.915421 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.915399    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "da84d4bd-9232-43b7-a8e8-fd2305d5135e" (UID: "da84d4bd-9232-43b7-a8e8-fd2305d5135e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:23:33.915521 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.915401    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da84d4bd-9232-43b7-a8e8-fd2305d5135e-kube-api-access-bl2ls" (OuterVolumeSpecName: "kube-api-access-bl2ls") pod "da84d4bd-9232-43b7-a8e8-fd2305d5135e" (UID: "da84d4bd-9232-43b7-a8e8-fd2305d5135e"). InnerVolumeSpecName "kube-api-access-bl2ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:23:33.915521 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:33.915430    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-dshm" (OuterVolumeSpecName: "dshm") pod "da84d4bd-9232-43b7-a8e8-fd2305d5135e" (UID: "da84d4bd-9232-43b7-a8e8-fd2305d5135e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:34.014963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.014934    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl2ls\" (UniqueName: \"kubernetes.io/projected/da84d4bd-9232-43b7-a8e8-fd2305d5135e-kube-api-access-bl2ls\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.014963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.014958    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da84d4bd-9232-43b7-a8e8-fd2305d5135e-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.014963 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.014968    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da84d4bd-9232-43b7-a8e8-fd2305d5135e-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.379720 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.379650    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8" podUID="2773fdef-e591-4142-a47a-b7ee30a26ea7" containerName="main" containerID="cri-o://d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e" gracePeriod=45
Jul 11 08:23:34.478569 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.478517    2572 generic.go:356] "Generic (PLEG): container finished" podID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" containerID="cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a" exitCode=0
Jul 11 08:23:34.478832 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.478586    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"
Jul 11 08:23:34.478832 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.478606    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" event={"ID":"da84d4bd-9232-43b7-a8e8-fd2305d5135e","Type":"ContainerDied","Data":"cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a"}
Jul 11 08:23:34.478832 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.478660    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" event={"ID":"da84d4bd-9232-43b7-a8e8-fd2305d5135e","Type":"ContainerDied","Data":"315ea2e2513821dab83508d49eebb34a0b9704096654d8e8d1a992a5619782f0"}
Jul 11 08:23:34.478832 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.478681    2572 scope.go:117] "RemoveContainer" containerID="cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a"
Jul 11 08:23:34.489941 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.489909    2572 scope.go:117] "RemoveContainer" containerID="cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a"
Jul 11 08:23:34.490235 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:23:34.490214    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a\": container with ID starting with cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a not found: ID does not exist" containerID="cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a"
Jul 11 08:23:34.490299 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.490243    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a"} err="failed to get container status \"cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a\": rpc error: code = NotFound desc = could not find container \"cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a\": container with ID starting with cfe0794238eb4dcc3a46c35b68f2b53914c02fe554c91a249de2944d8d741f4a not found: ID does not exist"
Jul 11 08:23:34.495285 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.495265    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"]
Jul 11 08:23:34.498784 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.498761    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj"]
Jul 11 08:23:34.628458 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.628438    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:23:34.721557 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721505    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn9gt\" (UniqueName: \"kubernetes.io/projected/2773fdef-e591-4142-a47a-b7ee30a26ea7-kube-api-access-bn9gt\") pod \"2773fdef-e591-4142-a47a-b7ee30a26ea7\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") "
Jul 11 08:23:34.721738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721574    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-model-cache\") pod \"2773fdef-e591-4142-a47a-b7ee30a26ea7\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") "
Jul 11 08:23:34.721738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721606    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-home\") pod \"2773fdef-e591-4142-a47a-b7ee30a26ea7\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") "
Jul 11 08:23:34.721738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721650    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-tmp-dir\") pod \"2773fdef-e591-4142-a47a-b7ee30a26ea7\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") "
Jul 11 08:23:34.721738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721675    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-dshm\") pod \"2773fdef-e591-4142-a47a-b7ee30a26ea7\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") "
Jul 11 08:23:34.721738 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721713    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2773fdef-e591-4142-a47a-b7ee30a26ea7-tls-certs\") pod \"2773fdef-e591-4142-a47a-b7ee30a26ea7\" (UID: \"2773fdef-e591-4142-a47a-b7ee30a26ea7\") "
Jul 11 08:23:34.721999 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721860    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-model-cache" (OuterVolumeSpecName: "model-cache") pod "2773fdef-e591-4142-a47a-b7ee30a26ea7" (UID: "2773fdef-e591-4142-a47a-b7ee30a26ea7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:34.721999 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721941    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-home" (OuterVolumeSpecName: "home") pod "2773fdef-e591-4142-a47a-b7ee30a26ea7" (UID: "2773fdef-e591-4142-a47a-b7ee30a26ea7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:34.721999 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721957    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2773fdef-e591-4142-a47a-b7ee30a26ea7" (UID: "2773fdef-e591-4142-a47a-b7ee30a26ea7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:34.721999 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.721997    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.723809 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.723786    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-dshm" (OuterVolumeSpecName: "dshm") pod "2773fdef-e591-4142-a47a-b7ee30a26ea7" (UID: "2773fdef-e591-4142-a47a-b7ee30a26ea7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:23:34.723918 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.723857    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2773fdef-e591-4142-a47a-b7ee30a26ea7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2773fdef-e591-4142-a47a-b7ee30a26ea7" (UID: "2773fdef-e591-4142-a47a-b7ee30a26ea7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:23:34.723918 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.723909    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2773fdef-e591-4142-a47a-b7ee30a26ea7-kube-api-access-bn9gt" (OuterVolumeSpecName: "kube-api-access-bn9gt") pod "2773fdef-e591-4142-a47a-b7ee30a26ea7" (UID: "2773fdef-e591-4142-a47a-b7ee30a26ea7"). InnerVolumeSpecName "kube-api-access-bn9gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:23:34.822490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.822413    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2773fdef-e591-4142-a47a-b7ee30a26ea7-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.822490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.822439    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn9gt\" (UniqueName: \"kubernetes.io/projected/2773fdef-e591-4142-a47a-b7ee30a26ea7-kube-api-access-bn9gt\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.822490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.822448    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.822490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.822457    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:34.822490 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:34.822466    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2773fdef-e591-4142-a47a-b7ee30a26ea7-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:23:35.486212 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.486181    2572 generic.go:356] "Generic (PLEG): container finished" podID="2773fdef-e591-4142-a47a-b7ee30a26ea7" containerID="d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e" exitCode=0
Jul 11 08:23:35.486389 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.486223    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"
Jul 11 08:23:35.486389 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.486230    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8" event={"ID":"2773fdef-e591-4142-a47a-b7ee30a26ea7","Type":"ContainerDied","Data":"d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e"}
Jul 11 08:23:35.486389 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.486257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8" event={"ID":"2773fdef-e591-4142-a47a-b7ee30a26ea7","Type":"ContainerDied","Data":"f8871eb61e2066b9a24fce50fe3a31242ff413e3c682fbf2a0d81a8fc55cb15f"}
Jul 11 08:23:35.486389 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.486273    2572 scope.go:117] "RemoveContainer" containerID="d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e"
Jul 11 08:23:35.495434 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.495417    2572 scope.go:117] "RemoveContainer" containerID="d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e"
Jul 11 08:23:35.495720 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:23:35.495702    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e\": container with ID starting with d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e not found: ID does not exist" containerID="d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e"
Jul 11 08:23:35.495780 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.495727    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e"} err="failed to get container status \"d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e\": rpc error: code = NotFound desc = could not find container \"d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e\": container with ID starting with d247dde88e6392da79e17dbe6307359c61f7c14e41c8e03a55ef8a35d8abbd2e not found: ID does not exist"
Jul 11 08:23:35.505933 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.505912    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"]
Jul 11 08:23:35.508956 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:35.508938    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/auth-test-model-12fff0-kserve-6c96685bd7-8rcj8"]
Jul 11 08:23:36.000977 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:36.000942    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:23:36.294820 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:36.294744    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2773fdef-e591-4142-a47a-b7ee30a26ea7" path="/var/lib/kubelet/pods/2773fdef-e591-4142-a47a-b7ee30a26ea7/volumes"
Jul 11 08:23:36.295140 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:36.295127    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" path="/var/lib/kubelet/pods/da84d4bd-9232-43b7-a8e8-fd2305d5135e/volumes"
Jul 11 08:23:38.178907 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:38.178871    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:38.178907 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:38.178910    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:38.190879 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:38.190854    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:38.513167 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:38.513091    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:23:38.791737 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:38.791643    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-f178a6/auth-test-model-f178a6-kserve-76fb9794cf-tvrjj" podUID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Jul 11 08:23:49.051711 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.051679    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"]
Jul 11 08:23:49.052160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.052046    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" containerName="main"
Jul 11 08:23:49.052160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.052056    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" containerName="main"
Jul 11 08:23:49.052160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.052073    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2773fdef-e591-4142-a47a-b7ee30a26ea7" containerName="main"
Jul 11 08:23:49.052160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.052079    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2773fdef-e591-4142-a47a-b7ee30a26ea7" containerName="main"
Jul 11 08:23:49.052160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.052138    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2773fdef-e591-4142-a47a-b7ee30a26ea7" containerName="main"
Jul 11 08:23:49.052160 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.052147    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="da84d4bd-9232-43b7-a8e8-fd2305d5135e" containerName="main"
Jul 11 08:23:49.056473 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.056457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.059590 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.059568    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"kube-root-ca.crt\""
Jul 11 08:23:49.059689 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.059621    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"default-dockercfg-kmvlk\""
Jul 11 08:23:49.059689 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.059623    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"openshift-service-ca.crt\""
Jul 11 08:23:49.059689 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.059657    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"sub-test-model-12fff0-kserve-self-signed-certs\""
Jul 11 08:23:49.064716 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.064695    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"]
Jul 11 08:23:49.074788 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.074764    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:23:49.149175 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.149144    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tmp-dir\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.149334 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.149190    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-model-cache\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.149334 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.149276    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnvpz\" (UniqueName: \"kubernetes.io/projected/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-kube-api-access-hnvpz\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.149334 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.149308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-home\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.149440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.149363    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tls-certs\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.149440 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.149390    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-dshm\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250488 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250457    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tls-certs\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250488 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250493    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-dshm\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250744 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250567    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tmp-dir\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250744 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-model-cache\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250744 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250677    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnvpz\" (UniqueName: \"kubernetes.io/projected/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-kube-api-access-hnvpz\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250744 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250707    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-home\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.250955 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250921    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tmp-dir\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.251014 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.250986    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-model-cache\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.251068 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.251031    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-home\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.252720 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.252696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-dshm\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.252954 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.252939    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tls-certs\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.258726 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.258701    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnvpz\" (UniqueName: \"kubernetes.io/projected/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-kube-api-access-hnvpz\") pod \"sub-test-model-12fff0-kserve-8444df57-td8f8\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") " pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.366338 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.366247    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:49.492629 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.492603    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"]
Jul 11 08:23:49.494787 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:23:49.494762    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39ee1645_dd1d_48f0_8a9d_28be0dbff3bf.slice/crio-e7ed8b5a2df3acaaebe4ec8738930026498ee289f698565938258580e56e97fb WatchSource:0}: Error finding container e7ed8b5a2df3acaaebe4ec8738930026498ee289f698565938258580e56e97fb: Status 404 returned error can't find the container with id e7ed8b5a2df3acaaebe4ec8738930026498ee289f698565938258580e56e97fb
Jul 11 08:23:49.556357 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.556326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8" event={"ID":"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf","Type":"ContainerStarted","Data":"3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899"}
Jul 11 08:23:49.556459 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.556365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8" event={"ID":"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf","Type":"ContainerStarted","Data":"e7ed8b5a2df3acaaebe4ec8738930026498ee289f698565938258580e56e97fb"}
Jul 11 08:23:49.573342 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:49.573288    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8" podStartSLOduration=0.573267038 podStartE2EDuration="573.267038ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:23:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:23:49.570717925 +0000 UTC m=+2735.774136787" watchObservedRunningTime="2026-07-11 08:23:49.573267038 +0000 UTC m=+2735.776685901"
Jul 11 08:23:59.366843 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:59.366793    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:59.366843 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:59.366853    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:59.379032 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:59.379009    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:23:59.614592 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:23:59.614566    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:24:14.474170 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:14.474141    2572 scope.go:117] "RemoveContainer" containerID="8d8d949d6882ca93f883b1b4237bb37dcea58c423f3522e3c5573ae1a945b4ae"
Jul 11 08:24:24.488010 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:24.487976    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"]
Jul 11 08:24:24.529096 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:24.529065    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:24:25.238173 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:25.238139    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"]
Jul 11 08:24:25.257116 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:25.257089    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:24:28.785620 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:28.785586    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:24:39.507627 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.507516    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4" podUID="79965395-e2f0-48ee-bcea-0726a0b97346" containerName="main" containerID="cri-o://728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1" gracePeriod=45
Jul 11 08:24:39.752941 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.752915    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:24:39.802119 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.802090    2572 generic.go:356] "Generic (PLEG): container finished" podID="79965395-e2f0-48ee-bcea-0726a0b97346" containerID="728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1" exitCode=0
Jul 11 08:24:39.802291 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.802130    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"
Jul 11 08:24:39.802291 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.802166    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4" event={"ID":"79965395-e2f0-48ee-bcea-0726a0b97346","Type":"ContainerDied","Data":"728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1"}
Jul 11 08:24:39.802291 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.802208    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4" event={"ID":"79965395-e2f0-48ee-bcea-0726a0b97346","Type":"ContainerDied","Data":"ce921b94e4547733ef54d6c8cc03a8961f2f9f025f5d45589d6c4ea800c584fe"}
Jul 11 08:24:39.802291 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.802226    2572 scope.go:117] "RemoveContainer" containerID="728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1"
Jul 11 08:24:39.810378 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810360    2572 scope.go:117] "RemoveContainer" containerID="728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1"
Jul 11 08:24:39.810597 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810578    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8x9x\" (UniqueName: \"kubernetes.io/projected/79965395-e2f0-48ee-bcea-0726a0b97346-kube-api-access-t8x9x\") pod \"79965395-e2f0-48ee-bcea-0726a0b97346\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") "
Jul 11 08:24:39.810660 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810617    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-tmp-dir\") pod \"79965395-e2f0-48ee-bcea-0726a0b97346\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") "
Jul 11 08:24:39.810708 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:24:39.810664    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1\": container with ID starting with 728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1 not found: ID does not exist" containerID="728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1"
Jul 11 08:24:39.810708 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810690    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1"} err="failed to get container status \"728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1\": rpc error: code = NotFound desc = could not find container \"728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1\": container with ID starting with 728fb32cd69d00a5a70e8f44838f88fc14ee79c1401b904f2af3897e1ada28f1 not found: ID does not exist"
Jul 11 08:24:39.810708 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810670    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-model-cache\") pod \"79965395-e2f0-48ee-bcea-0726a0b97346\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") "
Jul 11 08:24:39.810891 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810730    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-dshm\") pod \"79965395-e2f0-48ee-bcea-0726a0b97346\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") "
Jul 11 08:24:39.810891 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810785    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79965395-e2f0-48ee-bcea-0726a0b97346-tls-certs\") pod \"79965395-e2f0-48ee-bcea-0726a0b97346\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") "
Jul 11 08:24:39.810891 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-home\") pod \"79965395-e2f0-48ee-bcea-0726a0b97346\" (UID: \"79965395-e2f0-48ee-bcea-0726a0b97346\") "
Jul 11 08:24:39.810891 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810878    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-model-cache" (OuterVolumeSpecName: "model-cache") pod "79965395-e2f0-48ee-bcea-0726a0b97346" (UID: "79965395-e2f0-48ee-bcea-0726a0b97346"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:39.811096 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.810980    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79965395-e2f0-48ee-bcea-0726a0b97346" (UID: "79965395-e2f0-48ee-bcea-0726a0b97346"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:39.811192 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.811171    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:39.811251 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.811184    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-home" (OuterVolumeSpecName: "home") pod "79965395-e2f0-48ee-bcea-0726a0b97346" (UID: "79965395-e2f0-48ee-bcea-0726a0b97346"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:39.811251 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.811196    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:39.812752 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.812725    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79965395-e2f0-48ee-bcea-0726a0b97346-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79965395-e2f0-48ee-bcea-0726a0b97346" (UID: "79965395-e2f0-48ee-bcea-0726a0b97346"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:24:39.812851 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.812766    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-dshm" (OuterVolumeSpecName: "dshm") pod "79965395-e2f0-48ee-bcea-0726a0b97346" (UID: "79965395-e2f0-48ee-bcea-0726a0b97346"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:39.812851 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.812783    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79965395-e2f0-48ee-bcea-0726a0b97346-kube-api-access-t8x9x" (OuterVolumeSpecName: "kube-api-access-t8x9x") pod "79965395-e2f0-48ee-bcea-0726a0b97346" (UID: "79965395-e2f0-48ee-bcea-0726a0b97346"). InnerVolumeSpecName "kube-api-access-t8x9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:24:39.912717 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.912687    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79965395-e2f0-48ee-bcea-0726a0b97346-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:39.912717 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.912713    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:39.912717 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.912723    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8x9x\" (UniqueName: \"kubernetes.io/projected/79965395-e2f0-48ee-bcea-0726a0b97346-kube-api-access-t8x9x\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:39.912933 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:39.912731    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79965395-e2f0-48ee-bcea-0726a0b97346-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.129873 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.129841    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"]
Jul 11 08:24:40.135259 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.135228    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/sub-test-model-f178a6-kserve-77bcf89cb5-62ck4"]
Jul 11 08:24:40.258139 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.258094    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8" podUID="39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" containerName="main" containerID="cri-o://3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899" gracePeriod=45
Jul 11 08:24:40.294424 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.294392    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79965395-e2f0-48ee-bcea-0726a0b97346" path="/var/lib/kubelet/pods/79965395-e2f0-48ee-bcea-0726a0b97346/volumes"
Jul 11 08:24:40.503321 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.503300    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:24:40.619662 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619631    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tls-certs\") pod \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") "
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619680    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-dshm\") pod \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") "
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619699    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tmp-dir\") pod \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") "
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619720    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnvpz\" (UniqueName: \"kubernetes.io/projected/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-kube-api-access-hnvpz\") pod \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") "
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619742    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-home\") pod \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") "
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619839    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-model-cache\") pod \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\" (UID: \"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf\") "
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.619977    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" (UID: "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:40.620063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.620047    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-home" (OuterVolumeSpecName: "home") pod "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" (UID: "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:40.620370 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.620141    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" (UID: "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:40.620370 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.620164    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.620370 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.620186    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.621702 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.621674    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-dshm" (OuterVolumeSpecName: "dshm") pod "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" (UID: "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:24:40.621798 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.621705    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" (UID: "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:24:40.621798 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.621738    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-kube-api-access-hnvpz" (OuterVolumeSpecName: "kube-api-access-hnvpz") pod "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" (UID: "39ee1645-dd1d-48f0-8a9d-28be0dbff3bf"). InnerVolumeSpecName "kube-api-access-hnvpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:24:40.721489 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.721458    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.721489 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.721487    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.721489 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.721497    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnvpz\" (UniqueName: \"kubernetes.io/projected/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-kube-api-access-hnvpz\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.721723 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.721507    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:24:40.808989 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.808958    2572 generic.go:356] "Generic (PLEG): container finished" podID="39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" containerID="3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899" exitCode=0
Jul 11 08:24:40.809139 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.808994    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"
Jul 11 08:24:40.809139 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.809020    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8" event={"ID":"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf","Type":"ContainerDied","Data":"3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899"}
Jul 11 08:24:40.809139 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.809052    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8" event={"ID":"39ee1645-dd1d-48f0-8a9d-28be0dbff3bf","Type":"ContainerDied","Data":"e7ed8b5a2df3acaaebe4ec8738930026498ee289f698565938258580e56e97fb"}
Jul 11 08:24:40.809139 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.809072    2572 scope.go:117] "RemoveContainer" containerID="3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899"
Jul 11 08:24:40.818782 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.818767    2572 scope.go:117] "RemoveContainer" containerID="3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899"
Jul 11 08:24:40.818999 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:24:40.818983    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899\": container with ID starting with 3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899 not found: ID does not exist" containerID="3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899"
Jul 11 08:24:40.819051 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.819004    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899"} err="failed to get container status \"3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899\": rpc error: code = NotFound desc = could not find container \"3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899\": container with ID starting with 3ae61b21c644810a9360e7a1021a7dd36e5948df0d00943ec775aa0772006899 not found: ID does not exist"
Jul 11 08:24:40.831978 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.831957    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"]
Jul 11 08:24:40.834958 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:40.834938    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/sub-test-model-12fff0-kserve-8444df57-td8f8"]
Jul 11 08:24:42.294269 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:42.294236    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" path="/var/lib/kubelet/pods/39ee1645-dd1d-48f0-8a9d-28be0dbff3bf/volumes"
Jul 11 08:24:47.941823 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:47.941793    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:24:48.042686 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:48.042659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:24:52.642794 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.642766    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"]
Jul 11 08:24:52.643156 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.643147    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79965395-e2f0-48ee-bcea-0726a0b97346" containerName="main"
Jul 11 08:24:52.643156 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.643157    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79965395-e2f0-48ee-bcea-0726a0b97346" containerName="main"
Jul 11 08:24:52.643229 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.643168    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" containerName="main"
Jul 11 08:24:52.643229 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.643174    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" containerName="main"
Jul 11 08:24:52.643290 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.643233    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ee1645-dd1d-48f0-8a9d-28be0dbff3bf" containerName="main"
Jul 11 08:24:52.643290 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.643244    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="79965395-e2f0-48ee-bcea-0726a0b97346" containerName="main"
Jul 11 08:24:52.646205 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.646190    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.648387 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.648365    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"openshift-service-ca.crt\""
Jul 11 08:24:52.648692 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.648672    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"default-dockercfg-kmvlk\""
Jul 11 08:24:52.649645 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.649627    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"kube-root-ca.crt\""
Jul 11 08:24:52.649721 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.649681    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12fff0\"/\"rate-test-model-12fff0-kserve-self-signed-certs\""
Jul 11 08:24:52.654755 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.654737    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"]
Jul 11 08:24:52.724521 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.724469    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-tmp-dir\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.724732 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.724561    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-home\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.724732 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.724592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-dshm\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.724732 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.724614    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-model-cache\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.724732 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.724639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l75wt\" (UniqueName: \"kubernetes.io/projected/ff661527-9466-475d-b938-84b7c5a1d070-kube-api-access-l75wt\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.724732 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.724701    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff661527-9466-475d-b938-84b7c5a1d070-tls-certs\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.745192 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.745154    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:24:52.825594 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825569    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-tmp-dir\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.825737 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825606    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-home\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.825737 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825623    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-dshm\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.825737 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825637    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-model-cache\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.825737 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825652    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75wt\" (UniqueName: \"kubernetes.io/projected/ff661527-9466-475d-b938-84b7c5a1d070-kube-api-access-l75wt\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.825737 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff661527-9466-475d-b938-84b7c5a1d070-tls-certs\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.826012 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.825991    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-home\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.826063 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.826039    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-model-cache\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.826100 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.826076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-tmp-dir\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.827943 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.827929    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-dshm\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.828192 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.828176    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff661527-9466-475d-b938-84b7c5a1d070-tls-certs\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.832859 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.832840    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l75wt\" (UniqueName: \"kubernetes.io/projected/ff661527-9466-475d-b938-84b7c5a1d070-kube-api-access-l75wt\") pod \"rate-test-model-12fff0-kserve-58476d86c-vqgqn\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") " pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:52.956228 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:52.956140    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:24:53.084277 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:53.084253    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"]
Jul 11 08:24:53.086078 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:24:53.086050    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff661527_9466_475d_b938_84b7c5a1d070.slice/crio-fa0bb1da565e5862ec3277f2bba1d76089fd1cf77b50d46dc1d7e50edeed6a98 WatchSource:0}: Error finding container fa0bb1da565e5862ec3277f2bba1d76089fd1cf77b50d46dc1d7e50edeed6a98: Status 404 returned error can't find the container with id fa0bb1da565e5862ec3277f2bba1d76089fd1cf77b50d46dc1d7e50edeed6a98
Jul 11 08:24:53.873000 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:53.872966    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn" event={"ID":"ff661527-9466-475d-b938-84b7c5a1d070","Type":"ContainerStarted","Data":"855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858"}
Jul 11 08:24:53.873000 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:53.873004    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn" event={"ID":"ff661527-9466-475d-b938-84b7c5a1d070","Type":"ContainerStarted","Data":"fa0bb1da565e5862ec3277f2bba1d76089fd1cf77b50d46dc1d7e50edeed6a98"}
Jul 11 08:24:53.890449 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:24:53.890402    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn" podStartSLOduration=1.8903866009999999 podStartE2EDuration="1.890386601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:24:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:24:53.888727333 +0000 UTC m=+2800.092146195" watchObservedRunningTime="2026-07-11 08:24:53.890386601 +0000 UTC m=+2800.093805464"
Jul 11 08:25:02.956893 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:02.956856    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:25:02.957447 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:02.956931    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:25:02.969297 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:02.969273    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:25:03.938928 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:03.938894    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:25:39.951408 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:39.951374    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:25:40.689007 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:40.688973    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"]
Jul 11 08:25:41.127627 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:41.127589    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:25:55.708683 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.708618    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn" podUID="ff661527-9466-475d-b938-84b7c5a1d070" containerName="main" containerID="cri-o://855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858" gracePeriod=45
Jul 11 08:25:55.951748 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.951724    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:25:55.979460 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979388    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-model-cache\") pod \"ff661527-9466-475d-b938-84b7c5a1d070\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") "
Jul 11 08:25:55.979618 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979470    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-dshm\") pod \"ff661527-9466-475d-b938-84b7c5a1d070\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") "
Jul 11 08:25:55.979618 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979519    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff661527-9466-475d-b938-84b7c5a1d070-tls-certs\") pod \"ff661527-9466-475d-b938-84b7c5a1d070\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") "
Jul 11 08:25:55.979618 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979577    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l75wt\" (UniqueName: \"kubernetes.io/projected/ff661527-9466-475d-b938-84b7c5a1d070-kube-api-access-l75wt\") pod \"ff661527-9466-475d-b938-84b7c5a1d070\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") "
Jul 11 08:25:55.979618 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979605    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-tmp-dir\") pod \"ff661527-9466-475d-b938-84b7c5a1d070\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") "
Jul 11 08:25:55.979843 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979645    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-home\") pod \"ff661527-9466-475d-b938-84b7c5a1d070\" (UID: \"ff661527-9466-475d-b938-84b7c5a1d070\") "
Jul 11 08:25:55.979843 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979641    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff661527-9466-475d-b938-84b7c5a1d070" (UID: "ff661527-9466-475d-b938-84b7c5a1d070"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:25:55.979954 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979851    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff661527-9466-475d-b938-84b7c5a1d070" (UID: "ff661527-9466-475d-b938-84b7c5a1d070"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:25:55.979954 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979936    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-home" (OuterVolumeSpecName: "home") pod "ff661527-9466-475d-b938-84b7c5a1d070" (UID: "ff661527-9466-475d-b938-84b7c5a1d070"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:25:55.979954 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979948    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-tmp-dir\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:25:55.980371 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.979965    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-model-cache\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:25:55.981758 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.981720    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-dshm" (OuterVolumeSpecName: "dshm") pod "ff661527-9466-475d-b938-84b7c5a1d070" (UID: "ff661527-9466-475d-b938-84b7c5a1d070"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 08:25:55.981857 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.981800    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff661527-9466-475d-b938-84b7c5a1d070-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff661527-9466-475d-b938-84b7c5a1d070" (UID: "ff661527-9466-475d-b938-84b7c5a1d070"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 08:25:55.981857 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:55.981813    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff661527-9466-475d-b938-84b7c5a1d070-kube-api-access-l75wt" (OuterVolumeSpecName: "kube-api-access-l75wt") pod "ff661527-9466-475d-b938-84b7c5a1d070" (UID: "ff661527-9466-475d-b938-84b7c5a1d070"). InnerVolumeSpecName "kube-api-access-l75wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 08:25:56.080926 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.080891    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-dshm\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:25:56.080926 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.080919    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff661527-9466-475d-b938-84b7c5a1d070-tls-certs\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:25:56.080926 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.080932    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l75wt\" (UniqueName: \"kubernetes.io/projected/ff661527-9466-475d-b938-84b7c5a1d070-kube-api-access-l75wt\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:25:56.081142 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.080944    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff661527-9466-475d-b938-84b7c5a1d070-home\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:25:56.175256 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.175221    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff661527-9466-475d-b938-84b7c5a1d070" containerID="855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858" exitCode=0
Jul 11 08:25:56.175422 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.175266    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"
Jul 11 08:25:56.175422 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.175308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn" event={"ID":"ff661527-9466-475d-b938-84b7c5a1d070","Type":"ContainerDied","Data":"855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858"}
Jul 11 08:25:56.175422 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.175355    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn" event={"ID":"ff661527-9466-475d-b938-84b7c5a1d070","Type":"ContainerDied","Data":"fa0bb1da565e5862ec3277f2bba1d76089fd1cf77b50d46dc1d7e50edeed6a98"}
Jul 11 08:25:56.175422 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.175374    2572 scope.go:117] "RemoveContainer" containerID="855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858"
Jul 11 08:25:56.184513 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.184494    2572 scope.go:117] "RemoveContainer" containerID="855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858"
Jul 11 08:25:56.184794 ip-10-0-141-181 kubenswrapper[2572]: E0711 08:25:56.184774    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858\": container with ID starting with 855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858 not found: ID does not exist" containerID="855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858"
Jul 11 08:25:56.184866 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.184805    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858"} err="failed to get container status \"855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858\": rpc error: code = NotFound desc = could not find container \"855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858\": container with ID starting with 855993d0c78b017382bcb544f44d08f20c8989136f38e870e58b363faa100858 not found: ID does not exist"
Jul 11 08:25:56.197506 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.197475    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"]
Jul 11 08:25:56.203520 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.203494    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-12fff0/rate-test-model-12fff0-kserve-58476d86c-vqgqn"]
Jul 11 08:25:56.295396 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:56.295312    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff661527-9466-475d-b938-84b7c5a1d070" path="/var/lib/kubelet/pods/ff661527-9466-475d-b938-84b7c5a1d070/volumes"
Jul 11 08:25:59.544275 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.544241    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"]
Jul 11 08:25:59.544693 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.544678    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff661527-9466-475d-b938-84b7c5a1d070" containerName="main"
Jul 11 08:25:59.544693 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.544694    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff661527-9466-475d-b938-84b7c5a1d070" containerName="main"
Jul 11 08:25:59.544787 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.544777    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff661527-9466-475d-b938-84b7c5a1d070" containerName="main"
Jul 11 08:25:59.549229 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.549215    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:25:59.551859 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.551837    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 11 08:25:59.551983 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.551884    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\""
Jul 11 08:25:59.555488 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.555466    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"]
Jul 11 08:25:59.608538 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.608502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2588eb3-87c3-4796-9bfa-bf512dad90fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz\" (UID: \"e2588eb3-87c3-4796-9bfa-bf512dad90fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:25:59.709145 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.709114    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2588eb3-87c3-4796-9bfa-bf512dad90fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz\" (UID: \"e2588eb3-87c3-4796-9bfa-bf512dad90fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:25:59.709736 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.709717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2588eb3-87c3-4796-9bfa-bf512dad90fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz\" (UID: \"e2588eb3-87c3-4796-9bfa-bf512dad90fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:25:59.860931 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.860902    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:25:59.985917 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:25:59.985894    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"]
Jul 11 08:25:59.987737 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:25:59.987711    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2588eb3_87c3_4796_9bfa_bf512dad90fc.slice/crio-11a0df9f16f45e2482c406be53e66553437bef643cc046dec6e2a9a58d7c80f2 WatchSource:0}: Error finding container 11a0df9f16f45e2482c406be53e66553437bef643cc046dec6e2a9a58d7c80f2: Status 404 returned error can't find the container with id 11a0df9f16f45e2482c406be53e66553437bef643cc046dec6e2a9a58d7c80f2
Jul 11 08:26:00.144253 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:00.144148    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:00.201736 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:00.201678    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz" event={"ID":"e2588eb3-87c3-4796-9bfa-bf512dad90fc","Type":"ContainerStarted","Data":"11a0df9f16f45e2482c406be53e66553437bef643cc046dec6e2a9a58d7c80f2"}
Jul 11 08:26:00.250015 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:00.249992    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:01.207664 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:01.207629    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2588eb3-87c3-4796-9bfa-bf512dad90fc" containerID="492fa97d059deec55c332c3ab2cf96dac2ec5b7f5ce122612f8bb0a6f024cb82" exitCode=0
Jul 11 08:26:01.208094 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:01.207694    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz" event={"ID":"e2588eb3-87c3-4796-9bfa-bf512dad90fc","Type":"ContainerDied","Data":"492fa97d059deec55c332c3ab2cf96dac2ec5b7f5ce122612f8bb0a6f024cb82"}
Jul 11 08:26:02.341204 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:02.341181    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:26:02.430099 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:02.430066    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2588eb3-87c3-4796-9bfa-bf512dad90fc-maas-api-service-ca\") pod \"e2588eb3-87c3-4796-9bfa-bf512dad90fc\" (UID: \"e2588eb3-87c3-4796-9bfa-bf512dad90fc\") "
Jul 11 08:26:02.430414 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:02.430390    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2588eb3-87c3-4796-9bfa-bf512dad90fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2588eb3-87c3-4796-9bfa-bf512dad90fc" (UID: "e2588eb3-87c3-4796-9bfa-bf512dad90fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 08:26:02.531174 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:02.531090    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2588eb3-87c3-4796-9bfa-bf512dad90fc-maas-api-service-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:26:03.219214 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:03.219178    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz" event={"ID":"e2588eb3-87c3-4796-9bfa-bf512dad90fc","Type":"ContainerDied","Data":"11a0df9f16f45e2482c406be53e66553437bef643cc046dec6e2a9a58d7c80f2"}
Jul 11 08:26:03.219214 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:03.219205    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz"
Jul 11 08:26:03.219433 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:03.219259    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11a0df9f16f45e2482c406be53e66553437bef643cc046dec6e2a9a58d7c80f2"
Jul 11 08:26:10.465441 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:10.465339    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:10.568800 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:10.568770    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:10.704889 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:10.704813    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:14.629555 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:14.629495    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:20.300835 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:20.300802    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:20.408114 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:20.408074    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:25.283878 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.283843    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:25.509016 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.508982    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:25.618120 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.618091    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"]
Jul 11 08:26:25.618487 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.618474    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2588eb3-87c3-4796-9bfa-bf512dad90fc" containerName="revoke-keys"
Jul 11 08:26:25.618544 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.618488    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2588eb3-87c3-4796-9bfa-bf512dad90fc" containerName="revoke-keys"
Jul 11 08:26:25.618589 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.618580    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2588eb3-87c3-4796-9bfa-bf512dad90fc" containerName="revoke-keys"
Jul 11 08:26:25.621591 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.621576    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:25.623759 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.623739    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\""
Jul 11 08:26:25.623869 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.623776    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 11 08:26:25.627266 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.627242    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"]
Jul 11 08:26:25.732653 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.732617    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c72098f-ecf0-405a-ae86-adf0d6a7ccc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc\" (UID: \"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:25.833454 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.833427    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c72098f-ecf0-405a-ae86-adf0d6a7ccc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc\" (UID: \"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:25.834036 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.834016    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c72098f-ecf0-405a-ae86-adf0d6a7ccc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc\" (UID: \"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:25.933034 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:25.932948    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:26.058879 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:26.058856    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"]
Jul 11 08:26:26.061032 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:26:26.061004    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c72098f_ecf0_405a_ae86_adf0d6a7ccc3.slice/crio-7efceda36f5a03fcbd7ad3d40021c899e8727193ddb86f7a7b4f3e71bad7e3c5 WatchSource:0}: Error finding container 7efceda36f5a03fcbd7ad3d40021c899e8727193ddb86f7a7b4f3e71bad7e3c5: Status 404 returned error can't find the container with id 7efceda36f5a03fcbd7ad3d40021c899e8727193ddb86f7a7b4f3e71bad7e3c5
Jul 11 08:26:26.328380 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:26.328348    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc" event={"ID":"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3","Type":"ContainerStarted","Data":"5c5ca5fd679e26e1923b2f17d6432ac95ee9650b4283cd9b7557cdd8e320fc94"}
Jul 11 08:26:26.328380 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:26.328396    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc" event={"ID":"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3","Type":"ContainerStarted","Data":"7efceda36f5a03fcbd7ad3d40021c899e8727193ddb86f7a7b4f3e71bad7e3c5"}
Jul 11 08:26:26.343541 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:26.343476    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc" podStartSLOduration=1.343461593 podStartE2EDuration="1.343461593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:26:26.341668939 +0000 UTC m=+2892.545087802" watchObservedRunningTime="2026-07-11 08:26:26.343461593 +0000 UTC m=+2892.546880493"
Jul 11 08:26:27.334156 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:27.334121    2572 generic.go:356] "Generic (PLEG): container finished" podID="5c72098f-ecf0-405a-ae86-adf0d6a7ccc3" containerID="5c5ca5fd679e26e1923b2f17d6432ac95ee9650b4283cd9b7557cdd8e320fc94" exitCode=0
Jul 11 08:26:27.334156 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:27.334156    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc" event={"ID":"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3","Type":"ContainerDied","Data":"5c5ca5fd679e26e1923b2f17d6432ac95ee9650b4283cd9b7557cdd8e320fc94"}
Jul 11 08:26:28.472992 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:28.472967    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:28.559022 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:28.558993    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c72098f-ecf0-405a-ae86-adf0d6a7ccc3-maas-api-service-ca\") pod \"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3\" (UID: \"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3\") "
Jul 11 08:26:28.559312 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:28.559291    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c72098f-ecf0-405a-ae86-adf0d6a7ccc3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c72098f-ecf0-405a-ae86-adf0d6a7ccc3" (UID: "5c72098f-ecf0-405a-ae86-adf0d6a7ccc3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 08:26:28.659912 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:28.659831    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c72098f-ecf0-405a-ae86-adf0d6a7ccc3-maas-api-service-ca\") on node \"ip-10-0-141-181.ec2.internal\" DevicePath \"\""
Jul 11 08:26:29.344071 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:29.344040    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc" event={"ID":"5c72098f-ecf0-405a-ae86-adf0d6a7ccc3","Type":"ContainerDied","Data":"7efceda36f5a03fcbd7ad3d40021c899e8727193ddb86f7a7b4f3e71bad7e3c5"}
Jul 11 08:26:29.344263 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:29.344089    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7efceda36f5a03fcbd7ad3d40021c899e8727193ddb86f7a7b4f3e71bad7e3c5"
Jul 11 08:26:29.344263 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:29.344044    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc"
Jul 11 08:26:36.013468 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:36.013439    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:40.701815 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:40.701777    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:46.654039 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:46.653992    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:46.711271 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:46.711240    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:51.342267 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:51.342224    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:51.579105 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:51.579069    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:26:56.736961 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:56.736921    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gh2gz_69e057be-95e0-40e8-bb64-685fafa31d56/manager/0.log"
Jul 11 08:26:57.453331 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:57.453298    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-rgq2l_acbe2277-11bc-49cc-92e7-eb0483fbbecb/copy-manifests/0.log"
Jul 11 08:26:57.465122 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:57.465092    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-rgq2l_acbe2277-11bc-49cc-92e7-eb0483fbbecb/manager/0.log"
Jul 11 08:26:58.410084 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:58.410054    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-393a88-ptgkx_fd85e9e8-9b75-4a36-ae34-542e11271559/revoke-keys/0.log"
Jul 11 08:26:58.516982 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:58.516956    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-f92f11-mkkrr_d07a0011-68a9-4f7a-9818-daa3b78bcc30/revoke-keys/0.log"
Jul 11 08:26:58.836164 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:58.836136    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-59f9af-ltw52_7511eab5-d8b2-4314-a6a7-198038cd0058/revoke-keys/0.log"
Jul 11 08:26:59.259975 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:59.259889    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-f178a6-fn5pz_e2588eb3-87c3-4796-9bfa-bf512dad90fc/revoke-keys/0.log"
Jul 11 08:26:59.365645 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:26:59.365618    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-12fff0-r2cgc_5c72098f-ecf0-405a-ae86-adf0d6a7ccc3/revoke-keys/0.log"
Jul 11 08:27:00.947514 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:00.947472    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-q59s4_0e012a0b-96ff-45dd-85db-426a94966a90/manager/0.log"
Jul 11 08:27:01.057427 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:01.057397    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z69bp_ff579be5-8da7-4a10-adfe-e654d709ab3d/kuadrant-console-plugin/0.log"
Jul 11 08:27:01.514995 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:01.514961    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kz64m_dd5bbddd-f698-4b78-9bed-599d6e399040/manager/0.log"
Jul 11 08:27:02.091486 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:02.091453    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd5d4cc95-9vkxw_237570bd-ba2d-4cc3-ae35-1901addea7e4/kube-auth-proxy/0.log"
Jul 11 08:27:02.994049 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:02.994025    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-45z99_22b726d0-21a7-4cbf-8a42-d50b8b53a0f0/main/0.log"
Jul 11 08:27:03.216271 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:03.216230    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ps6rr_09f906ab-adce-44de-a9c7-ca67bbeba4cf/main/0.log"
Jul 11 08:27:03.538088 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:03.538061    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-h5fqx_5b7d2e40-9534-402f-8802-f5aacd7ae8f7/main/0.log"
Jul 11 08:27:07.179972 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:07.179942    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:27:07.244055 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:07.244022    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:27:09.883036 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:09.883001    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pmtjg_ebe9b4a8-f4d9-4407-8874-767a013e4ab4/global-pull-secret-syncer/0.log"
Jul 11 08:27:09.968300 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:09.968269    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log"
Jul 11 08:27:10.088298 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:10.088267    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-181.ec2.internal_6f661395125ffa7bd7cd96cb8f2630ea/haproxy/0.log"
Jul 11 08:27:15.721727 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:15.721694    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-q59s4_0e012a0b-96ff-45dd-85db-426a94966a90/manager/0.log"
Jul 11 08:27:15.744913 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:15.744884    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z69bp_ff579be5-8da7-4a10-adfe-e654d709ab3d/kuadrant-console-plugin/0.log"
Jul 11 08:27:16.052688 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:16.052655    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kz64m_dd5bbddd-f698-4b78-9bed-599d6e399040/manager/0.log"
Jul 11 08:27:17.818536 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:17.818476    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-7t75d_1c488d3c-413b-4146-a23a-8403f16df1e9/cluster-monitoring-operator/0.log"
Jul 11 08:27:17.941058 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:17.941029    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-cfb6h_cac52812-189d-4b58-ad69-2ed05defb2fc/monitoring-plugin/0.log"
Jul 11 08:27:17.968805 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:17.968776    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cqp6t_edebb51e-3a83-4eb7-b990-a6e2c90786f1/node-exporter/0.log"
Jul 11 08:27:17.989547 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:17.989508    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cqp6t_edebb51e-3a83-4eb7-b990-a6e2c90786f1/kube-rbac-proxy/0.log"
Jul 11 08:27:18.017834 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.017811    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cqp6t_edebb51e-3a83-4eb7-b990-a6e2c90786f1/init-textfile/0.log"
Jul 11 08:27:18.283839 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.283731    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/prometheus/0.log"
Jul 11 08:27:18.338988 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.338961    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/config-reloader/0.log"
Jul 11 08:27:18.360688 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.360658    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/thanos-sidecar/0.log"
Jul 11 08:27:18.379559 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.379514    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/kube-rbac-proxy-web/0.log"
Jul 11 08:27:18.398456 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.398411    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/kube-rbac-proxy/0.log"
Jul 11 08:27:18.424019 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.423989    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/kube-rbac-proxy-thanos/0.log"
Jul 11 08:27:18.443084 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.443064    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8a224f6-56ca-4dea-a3f5-8cec3e932390/init-config-reloader/0.log"
Jul 11 08:27:18.468891 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.468860    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-ld2gf_09d813e4-f9c0-4180-9b5d-1d0089a78c34/prometheus-operator/0.log"
Jul 11 08:27:18.485360 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.485332    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-ld2gf_09d813e4-f9c0-4180-9b5d-1d0089a78c34/kube-rbac-proxy/0.log"
Jul 11 08:27:18.505569 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.505539    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-2dkrv_e284dcd0-1354-43df-88a8-a7b6a1e8c6de/prometheus-operator-admission-webhook/0.log"
Jul 11 08:27:18.609195 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.609168    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dbb775d55-l7slr_afa95054-ea6f-4f5f-86a0-011942972008/thanos-query/0.log"
Jul 11 08:27:18.627990 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.627965    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dbb775d55-l7slr_afa95054-ea6f-4f5f-86a0-011942972008/kube-rbac-proxy-web/0.log"
Jul 11 08:27:18.646632 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.646595    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dbb775d55-l7slr_afa95054-ea6f-4f5f-86a0-011942972008/kube-rbac-proxy/0.log"
Jul 11 08:27:18.664609 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.664579    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dbb775d55-l7slr_afa95054-ea6f-4f5f-86a0-011942972008/prom-label-proxy/0.log"
Jul 11 08:27:18.681849 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.681820    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dbb775d55-l7slr_afa95054-ea6f-4f5f-86a0-011942972008/kube-rbac-proxy-rules/0.log"
Jul 11 08:27:18.700280 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:18.700256    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dbb775d55-l7slr_afa95054-ea6f-4f5f-86a0-011942972008/kube-rbac-proxy-metrics/0.log"
Jul 11 08:27:19.406239 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.406203    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"]
Jul 11 08:27:19.406777 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.406750    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c72098f-ecf0-405a-ae86-adf0d6a7ccc3" containerName="revoke-keys"
Jul 11 08:27:19.406777 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.406776    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c72098f-ecf0-405a-ae86-adf0d6a7ccc3" containerName="revoke-keys"
Jul 11 08:27:19.406947 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.406875    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c72098f-ecf0-405a-ae86-adf0d6a7ccc3" containerName="revoke-keys"
Jul 11 08:27:19.410152 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.410132    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.412696 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.412674    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dchrv\"/\"default-dockercfg-xfdth\""
Jul 11 08:27:19.412795 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.412674    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dchrv\"/\"kube-root-ca.crt\""
Jul 11 08:27:19.413405 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.413385    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dchrv\"/\"openshift-service-ca.crt\""
Jul 11 08:27:19.419412 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.419388    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"]
Jul 11 08:27:19.494866 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.494828    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-lib-modules\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.494866 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.494868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-sys\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.495079 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.494922    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-proc\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.495079 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.494966    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88n84\" (UniqueName: \"kubernetes.io/projected/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-kube-api-access-88n84\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.495079 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.494989    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-podres\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596221 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-lib-modules\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596221 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596224    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-sys\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596474 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596268    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-proc\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596474 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88n84\" (UniqueName: \"kubernetes.io/projected/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-kube-api-access-88n84\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596474 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-podres\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596474 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596369    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-sys\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596474 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596371    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-proc\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596474 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596371    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-lib-modules\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.596719 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.596477    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-podres\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.604051 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.604026    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88n84\" (UniqueName: \"kubernetes.io/projected/c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab-kube-api-access-88n84\") pod \"perf-node-gather-daemonset-f2tsn\" (UID: \"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.720454 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.720348    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:19.856031 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:19.855996    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"]
Jul 11 08:27:19.859491 ip-10-0-141-181 kubenswrapper[2572]: W0711 08:27:19.859460    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc9ba04b7_0732_4f1b_a7f4_ec7d3a5089ab.slice/crio-d3a36bc736f066608cdd3acaf296beca6c588d9876840b03b02fe235d780ce62 WatchSource:0}: Error finding container d3a36bc736f066608cdd3acaf296beca6c588d9876840b03b02fe235d780ce62: Status 404 returned error can't find the container with id d3a36bc736f066608cdd3acaf296beca6c588d9876840b03b02fe235d780ce62
Jul 11 08:27:20.584275 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:20.584242    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn" event={"ID":"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab","Type":"ContainerStarted","Data":"eb4272b916c0df6fd5a5292bfccfff9afaa2b20b991f8cc4a8771ae1c2ad424b"}
Jul 11 08:27:20.584789 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:20.584297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn" event={"ID":"c9ba04b7-0732-4f1b-a7f4-ec7d3a5089ab","Type":"ContainerStarted","Data":"d3a36bc736f066608cdd3acaf296beca6c588d9876840b03b02fe235d780ce62"}
Jul 11 08:27:20.584789 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:20.584352    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:20.600324 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:20.600277    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn" podStartSLOduration=1.6002618979999998 podStartE2EDuration="1.600261898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 08:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:27:20.599019304 +0000 UTC m=+2946.802438167" watchObservedRunningTime="2026-07-11 08:27:20.600261898 +0000 UTC m=+2946.803680762"
Jul 11 08:27:20.756253 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:20.756222    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5647f495c5-9fqnd_8425ea90-c1bc-4889-a6df-910e8732c627/console/0.log"
Jul 11 08:27:20.788945 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:20.788913    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-nmbc4_6be5391d-fe71-4a51-b7e0-79f9c31df6ff/download-server/0.log"
Jul 11 08:27:21.285126 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:21.285093    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-vrnd7_c6b3de6a-29b0-4b25-81cb-22080bbfd51f/volume-data-source-validator/0.log"
Jul 11 08:27:22.219150 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:22.219122    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fbmfs_b0367759-a5db-44e9-aa69-2baec901fd6a/dns/0.log"
Jul 11 08:27:22.237226 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:22.237206    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fbmfs_b0367759-a5db-44e9-aa69-2baec901fd6a/kube-rbac-proxy/0.log"
Jul 11 08:27:22.360623 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:22.360599    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jt4gq_547f78c4-01e1-4826-8413-0685d61c4f36/dns-node-resolver/0.log"
Jul 11 08:27:22.913353 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:22.913325    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9gcz_fde0f7b1-e969-480a-8209-31641ab1966e/node-ca/0.log"
Jul 11 08:27:23.983087 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:23.983061    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd5d4cc95-9vkxw_237570bd-ba2d-4cc3-ae35-1901addea7e4/kube-auth-proxy/0.log"
Jul 11 08:27:24.868441 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:24.868412    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6rnj_62effbbf-9d06-4b9e-a9f0-95d23f91a8d8/serve-healthcheck-canary/0.log"
Jul 11 08:27:25.416748 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:25.416722    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h4nnq_8cba86ef-a175-40d8-ae06-54eed94ff4a5/kube-rbac-proxy/0.log"
Jul 11 08:27:25.434841 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:25.434809    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h4nnq_8cba86ef-a175-40d8-ae06-54eed94ff4a5/exporter/0.log"
Jul 11 08:27:25.453754 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:25.453735    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h4nnq_8cba86ef-a175-40d8-ae06-54eed94ff4a5/extractor/0.log"
Jul 11 08:27:26.600441 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:26.600415    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-f2tsn"
Jul 11 08:27:27.697040 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:27.697006    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gh2gz_69e057be-95e0-40e8-bb64-685fafa31d56/manager/0.log"
Jul 11 08:27:28.154621 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:28.154593    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-rgq2l_acbe2277-11bc-49cc-92e7-eb0483fbbecb/manager/0.log"
Jul 11 08:27:28.171881 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:28.171852    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-rgq2l_acbe2277-11bc-49cc-92e7-eb0483fbbecb/copy-manifests/0.log"
Jul 11 08:27:29.500127 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:29.500038    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-697d57cd68-2wm5n_9b049884-7d4d-40ca-a508-1d294895d351/manager/0.log"
Jul 11 08:27:29.727844 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:29.727818    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd5d4cc95-9vkxw_237570bd-ba2d-4cc3-ae35-1901addea7e4/kube-auth-proxy/0.log"
Jul 11 08:27:31.300460 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:31.300434    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd5d4cc95-9vkxw_237570bd-ba2d-4cc3-ae35-1901addea7e4/kube-auth-proxy/0.log"
Jul 11 08:27:34.332799 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:34.332773    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-tjdnr_8652c53f-c34f-4274-b2a9-6782aea58a8a/migrator/0.log"
Jul 11 08:27:34.354668 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:34.354643    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-tjdnr_8652c53f-c34f-4274-b2a9-6782aea58a8a/graceful-termination/0.log"
Jul 11 08:27:35.798111 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.798040    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:27:35.838711 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.838684    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/kube-multus-additional-cni-plugins/0.log"
Jul 11 08:27:35.862584 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.862551    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crg4g_9375fbe2-bd17-4abd-9ed0-9c9584f60ecd/konnectivity-agent/0.log" limit=1024
Jul 11 08:27:35.868497 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.868474    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/egress-router-binary-copy/0.log"
Jul 11 08:27:35.887692 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.887669    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/cni-plugins/0.log"
Jul 11 08:27:35.906739 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.906718    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/bond-cni-plugin/0.log"
Jul 11 08:27:35.924940 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.924920    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/routeoverride-cni/0.log"
Jul 11 08:27:35.945908 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.945889    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/whereabouts-cni-bincopy/0.log"
Jul 11 08:27:35.963609 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.963592    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lh56c_453778fe-8de5-49f4-b6dd-54592e220864/whereabouts-cni/0.log"
Jul 11 08:27:35.992932 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:35.992910    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d4gvg_9c4eea64-5cb7-45d0-8d2f-2e5f48d091bd/kube-multus/0.log"
Jul 11 08:27:36.075202 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:36.075177    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jnj7r_29f420da-3869-4a78-97a2-c4e78477cfb2/network-metrics-daemon/0.log"
Jul 11 08:27:36.091736 ip-10-0-141-181 kubenswrapper[2572]: I0711 08:27:36.091714    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jnj7r_29f420da-3869-4a78-97a2-c4e78477cfb2/kube-rbac-proxy/0.log"