Jul 18 13:32:12.212951 ip-10-0-135-180 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.622040    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626565    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626576    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626580    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626583    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626587    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626591    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626594    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:12.637572 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626598    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626601    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626603    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626606    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626609    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626611    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626614    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626617    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626620    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626622    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626625    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626628    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626637    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626640    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626642    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626645    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626648    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626651    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626654    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:12.649201 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626656    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626659    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626663    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626689    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626693    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626696    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626699    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626702    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626705    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626714    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626718    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626722    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626725    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626729    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626733    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626737    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626741    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626745    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626749    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626753    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:12.652749 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626757    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626761    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626765    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626769    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626773    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626777    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626782    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626786    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626791    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626795    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626799    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626803    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626807    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626811    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626815    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626820    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626824    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626828    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626832    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:12.664220 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626836    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626840    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626845    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626849    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626852    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626856    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626860    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626868    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626874    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626879    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626883    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626886    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626888    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626892    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626894    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626897    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626900    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626904    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626909    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:12.668241 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626911    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626914    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626917    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626920    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626922    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626925    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626928    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626931    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626933    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626936    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626939    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626941    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626944    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626947    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.626949    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627635    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627642    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627645    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627648    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627651    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:12.685255 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627653    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627656    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627659    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627662    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627664    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627685    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627689    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627692    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627694    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627697    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627700    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627703    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627705    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627708    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627712    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627714    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627717    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627719    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627723    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627726    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:12.687534 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627728    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627730    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627733    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627736    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627739    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627741    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627744    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627746    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627749    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627753    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627757    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627760    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627763    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627766    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627769    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627773    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627776    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627779    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627782    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:12.722081 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627784    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:12.695654 ip-10-0-135-180 systemd[1]: Started Kubernetes Kubelet.
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627805    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627812    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627816    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627819    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627822    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627824    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627827    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627831    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627834    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627836    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627839    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627846    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627848    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627851    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627854    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627858    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627861    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627864    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:12.724548 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627867    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627870    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627873    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627876    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627881    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627903    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627924    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627928    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627932    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627935    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627938    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627941    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627944    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627947    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627950    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627952    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627955    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627958    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627962    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627965    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:12.729012 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627967    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627970    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627973    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627975    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627978    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627981    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627984    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627986    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627989    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627992    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627995    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.627997    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628000    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628002    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628005    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628008    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628387    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628392    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628395    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628398    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:12.899581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628401    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628404    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628407    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628410    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628413    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628415    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628418    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628421    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628423    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628426    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628429    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628431    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628434    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628437    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628439    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628442    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628445    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628447    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628450    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628452    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:12.900435 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628455    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628457    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628460    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628463    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628465    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628468    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628471    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628473    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628476    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628479    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628482    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628486    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628489    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628492    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628494    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628497    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628500    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628503    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628506    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628508    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:12.901788 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628511    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628513    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628515    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628518    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628521    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628523    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628526    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628528    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628531    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628533    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628536    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628538    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628543    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628547    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628550    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628553    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628556    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628558    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628561    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628564    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:12.914366 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628567    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628569    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628572    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628579    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628582    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628585    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628587    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628590    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628593    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628595    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628597    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628600    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628603    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628605    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628608    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628611    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628613    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628616    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628620    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:12.916880 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628624    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628627    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628629    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628633    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628637    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628641    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628646    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628648    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628651    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628654    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628656    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628659    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628662    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628664    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628681    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.628684    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628759    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628769    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628779    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628786    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628792    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 13:32:12.922268 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628795    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628799    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628803    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628806    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628810    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628813    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628816    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628820    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628823    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628825    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628829    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628832    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628835    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628839    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628841    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628845    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628848    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628853    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628857    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628860    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628863    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628866    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628869    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628872    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 13:32:12.923004 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628875    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628878    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628881    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628886    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628889    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628892    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628896    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628899    2579 flags.go:64] FLAG: --enable-server="true"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628902    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628907    2579 flags.go:64] FLAG: --event-burst="100"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628910    2579 flags.go:64] FLAG: --event-qps="50"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628913    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628916    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628919    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628923    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628926    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628929    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628932    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628935    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628938    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628941    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628944    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628947    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628950    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628953    2579 flags.go:64] FLAG: --feature-gates=""
Jul 18 13:32:12.925879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628958    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628961    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628964    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628967    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628970    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628973    2579 flags.go:64] FLAG: --help="false"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628976    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-135-180.ec2.internal"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628979    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628982    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628985    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628990    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628993    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628996    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.628999    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629003    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629006    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629009    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629012    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629015    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629018    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629021    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629024    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629027    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629029    2579 flags.go:64] FLAG: --lock-file=""
Jul 18 13:32:12.926965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629032    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629035    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629038    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629044    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629046    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629049    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629052    2579 flags.go:64] FLAG: --logging-format="text"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629055    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629060    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629062    2579 flags.go:64] FLAG: --manifest-url=""
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629065    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629077    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629080    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629085    2579 flags.go:64] FLAG: --max-pods="110"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629088    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629091    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629094    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629097    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629101    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629104    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629107    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629113    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629116    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629120    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 13:32:13.022899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629123    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629126    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629131    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629134    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629137    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629140    2579 flags.go:64] FLAG: --port="10250"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629143    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629146    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cfea46ff5240d7b6"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629149    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629152    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629155    2579 flags.go:64] FLAG: --register-node="true"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629158    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629162    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629165    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629167    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629170    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629175    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629178    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629181    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629184    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629187    2579 flags.go:64] FLAG: --runonce="false"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629189    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629192    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629195    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629198    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 13:32:13.024265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629201    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629204    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629208    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629211    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629214    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629216    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629219    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629222    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629225    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629228    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629231    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629236    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629239    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629242    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629246    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629248    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629251    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629254    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629257    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629260    2579 flags.go:64] FLAG: --v="2"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629264    2579 flags.go:64] FLAG: --version="false"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629268    2579 flags.go:64] FLAG: --vmodule=""
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629272    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.629277    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629387    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:13.025011 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629390    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629393    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629396    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629399    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629402    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629404    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629407    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629410    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629413    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629417    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629419    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629422    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629424    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629427    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629430    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629434    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629436    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629439    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629442    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629444    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:13.026763 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629447    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629449    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629454    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629458    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629461    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629465    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629467    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629470    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629472    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629476    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629481    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629484    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629486    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629489    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629492    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629494    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629497    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629500    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629502    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:13.027469 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629505    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629507    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629511    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629513    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629516    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629518    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629521    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629524    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629526    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629529    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629532    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629535    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629538    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629540    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629543    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629545    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629548    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629550    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629553    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629556    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:13.028551 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629558    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629561    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629563    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629567    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629569    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629572    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629575    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629577    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629580    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629583    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629585    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629588    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629590    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629593    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629597    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629599    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629602    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629605    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629607    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:13.029280 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629610    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629613    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629615    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629618    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629621    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629623    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629626    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629628    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629631    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629633    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629636    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629639    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629642    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629644    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629647    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629650    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629653    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629656    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629658    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:13.030360 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.629661    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.630499    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.637287    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.637303    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.637323    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.637328    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637379    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637385    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637388    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637391    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637394    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637397    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637403    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637406    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637409    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:13.032153 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637412    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637414    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637417    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637420    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637423    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637426    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637429    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637432    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637434    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637437    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637439    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637442    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637445    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637447    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637450    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637452    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637455    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637457    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637460    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637464    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:13.032581 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637468    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637472    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637475    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637478    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637481    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637484    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637486    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637489    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637492    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637497    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637500    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637503    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637505    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637508    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637511    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637513    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637516    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637519    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637522    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:13.033089 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637525    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637527    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637530    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637532    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637534    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637537    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637540    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637542    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637544    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637547    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637550    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637552    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637555    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637558    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637560    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637563    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637566    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637569    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637571    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637574    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:13.033554 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637576    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637579    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637582    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637585    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637588    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637591    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637593    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637596    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637599    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637601    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637604    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637608    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637610    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637612    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637615    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637617    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637620    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637622    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637625    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637627    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:13.034055 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637630    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637633    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637636    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637641    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637645    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637649    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637652    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637655    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637657    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637660    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637662    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.637681    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637804    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637809    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:13.034540 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637812    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637816    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637819    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637822    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637825    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637828    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637831    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637834    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637837    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637839    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637842    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637845    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637847    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637850    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637853    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637855    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637858    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637862    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637866    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:13.034909 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637868    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637871    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637874    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637877    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637881    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637884    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637887    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637889    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637892    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637895    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637897    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637900    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637903    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637905    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637908    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637911    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637913    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637916    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637919    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637922    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:13.035364 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637924    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637927    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637929    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637932    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637935    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637937    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637940    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637942    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637945    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637947    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637951    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637955    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637957    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637960    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637963    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637965    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637969    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637971    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637974    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637976    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:13.035899 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637979    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637982    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637984    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637987    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637989    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637992    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637996    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.637999    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638002    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638005    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638008    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638011    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638014    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638017    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638020    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638022    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638025    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638028    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638031    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:13.036396 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638034    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638036    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638039    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638041    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638044    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638046    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638049    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638051    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638054    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638058    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638060    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638063    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638065    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638068    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638071    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638073    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638076    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638078    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:13.036876 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:12.638082    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.638087    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.638180    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.640556    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.641689    2579 server.go:1019] "Starting client certificate rotation"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.641788    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.641819    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.644908    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.646136    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.660872    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.663363    2579 log.go:25] "Validated CRI v1 image API"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.664872    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 13:32:13.037321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.669348    2579 fs.go:135] Filesystem UUIDs: map[4d76c625-a25e-4810-92b5-f0c91631ec02:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c75c6d58-1864-47ba-bd7b-6c589c984058:/dev/nvme0n1p3]
Jul 18 13:32:13.037624 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.669362    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 13:32:13.037624 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.672774    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.675218    2579 manager.go:216] Machine: {Timestamp:2026-07-18 13:32:12.673916832 +0000 UTC m=+0.343366528 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100838 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24cc6df65956df36b83836bb2d8eea SystemUUID:ec24cc6d-f659-56df-36b8-3836bb2d8eea BootID:84ab0aaf-1609-4a99-ae8d-617bb45bbb55 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e1:2e:11:c5:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e1:2e:11:c5:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:7b:2e:9f:f9:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.675540    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.675704    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.679349    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.679522    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.680677    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.680688    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.680696    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.680707    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.681953    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684357    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684461    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684487    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684496    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684509    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684524    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684533    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 13:32:13.037698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684848    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.684871    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.687518    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.689460    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.689488    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691160    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691173    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691179    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691185    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691193    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691199    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691206    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691211    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691217    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691222    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691230    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.691236    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.692075    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.692082    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.695079    2579 server.go:1268] "Started kubelet"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.695398    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.696728    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.696806    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.696867    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.698604    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.702717    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.704382    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.704427    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.705392    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.705393    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.705517    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.704803    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-180.ec2.internal.18c3658cc80c3fa7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-180.ec2.internal,UID:ip-10-0-135-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-180.ec2.internal,},FirstTimestamp:2026-07-18 13:32:12.694921127 +0000 UTC m=+0.364370823,LastTimestamp:2026-07-18 13:32:12.694921127 +0000 UTC m=+0.364370823,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-180.ec2.internal,}"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706182    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706196    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706390    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706419    2579 factory.go:55] Registering systemd factory
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706438    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706697    2579 factory.go:153] Registering CRI-O factory
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706713    2579 factory.go:223] Registration of the crio container factory successfully
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706783    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706807    2579 factory.go:103] Registering Raw factory
Jul 18 13:32:13.038173 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.706822    2579 manager.go:1196] Started watching for new ooms in manager
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.707046    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.707055    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.707200    2579 manager.go:349] Starting recovery of all containers
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.709926    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.710108    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.715327    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.715527    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.719224    2579 manager.go:354] Recovery completed
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.720866    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wh6j4"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.723480    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725102    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725127    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725137    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725662    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725684    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725686    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.725702    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.726354    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.726380    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.726390    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.726907    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-180.ec2.internal.18c3658cc9d8f619  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-180.ec2.internal,UID:ip-10-0-135-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-180.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-180.ec2.internal,},FirstTimestamp:2026-07-18 13:32:12.725114393 +0000 UTC m=+0.394564090,LastTimestamp:2026-07-18 13:32:12.725114393 +0000 UTC m=+0.394564090,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-180.ec2.internal,}"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.727047    2579 policy_none.go:49] "None policy: Start"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.727063    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.727076    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.727578    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wh6j4"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.728981    2579 policy_none.go:47] "Start"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756286    2579 manager.go:337] "Starting Device Plugin manager"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.756323    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756336    2579 server.go:87] "Starting device plugin registration server"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756552    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756563    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756725    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756785    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.756792    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.758087    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.758136    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.840681    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.840846    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.840868    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 13:32:13.039195 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.840907    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.843031    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.856753    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.857535    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.857568    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.857586    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.857608    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.868092    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.868110    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-180.ec2.internal\": node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.896624    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.941853    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"]
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.941906    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.942759    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.942790    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.942804    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944096    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944259    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944765    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944784    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944793    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944942    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944966    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.944975    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.945817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.945841    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.946436    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.946467    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:12.946476    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:12.997388    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.008740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d0e14a09e5c2e1fbdfe8ed18e98725a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal\" (UID: \"2d0e14a09e5c2e1fbdfe8ed18e98725a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.008762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d0e14a09e5c2e1fbdfe8ed18e98725a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal\" (UID: \"2d0e14a09e5c2e1fbdfe8ed18e98725a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.040201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.008781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34bcd3343349a9ff2d90b37e5a2ddd1b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-180.ec2.internal\" (UID: \"34bcd3343349a9ff2d90b37e5a2ddd1b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.097605 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.097585    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.108342 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.108323    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-180.ec2.internal\" not found" node="ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.108861 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.108841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d0e14a09e5c2e1fbdfe8ed18e98725a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal\" (UID: \"2d0e14a09e5c2e1fbdfe8ed18e98725a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.108961 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.108869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d0e14a09e5c2e1fbdfe8ed18e98725a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal\" (UID: \"2d0e14a09e5c2e1fbdfe8ed18e98725a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.108961 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.108921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34bcd3343349a9ff2d90b37e5a2ddd1b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-180.ec2.internal\" (UID: \"34bcd3343349a9ff2d90b37e5a2ddd1b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.108961 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.108928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d0e14a09e5c2e1fbdfe8ed18e98725a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal\" (UID: \"2d0e14a09e5c2e1fbdfe8ed18e98725a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.108961 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.108937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d0e14a09e5c2e1fbdfe8ed18e98725a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal\" (UID: \"2d0e14a09e5c2e1fbdfe8ed18e98725a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.108961 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.108953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34bcd3343349a9ff2d90b37e5a2ddd1b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-180.ec2.internal\" (UID: \"34bcd3343349a9ff2d90b37e5a2ddd1b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.111404 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.111388    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.112450 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.112436    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-180.ec2.internal\" not found" node="ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.115471 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.115455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.197661 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.197631    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.297813 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.297761    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.398711 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.398301    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.499820 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.499791    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.600355 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.600331    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.642411 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.642388    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 13:32:13.642833 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.642542    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 13:32:13.674722 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:13.674676    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bcd3343349a9ff2d90b37e5a2ddd1b.slice/crio-52ece0171dd6e9277004d07bfa6f90117e02a8db0560b12a4ab9ab5af01e4d3b WatchSource:0}: Error finding container 52ece0171dd6e9277004d07bfa6f90117e02a8db0560b12a4ab9ab5af01e4d3b: Status 404 returned error can't find the container with id 52ece0171dd6e9277004d07bfa6f90117e02a8db0560b12a4ab9ab5af01e4d3b
Jul 18 13:32:13.681314 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.681296    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 13:32:13.687147 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:13.687127    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0e14a09e5c2e1fbdfe8ed18e98725a.slice/crio-b0b0aa10764e27043164abda1c4e4e44bb74b727b117b8dc4d909fb493d9986e WatchSource:0}: Error finding container b0b0aa10764e27043164abda1c4e4e44bb74b727b117b8dc4d909fb493d9986e: Status 404 returned error can't find the container with id b0b0aa10764e27043164abda1c4e4e44bb74b727b117b8dc4d909fb493d9986e
Jul 18 13:32:13.700434 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:13.700413    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-180.ec2.internal\" not found"
Jul 18 13:32:13.731576 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.731544    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 13:27:13 +0000 UTC" deadline="2028-01-21 10:42:35.382039474 +0000 UTC"
Jul 18 13:32:13.731576 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.731572    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13245h10m21.650472324s"
Jul 18 13:32:13.783933 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.783907    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:13.806652 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.806634    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.815072 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.815027    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 13:32:13.816985 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.816972    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal"
Jul 18 13:32:13.828379 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.828364    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 13:32:13.844318 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.844269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal" event={"ID":"34bcd3343349a9ff2d90b37e5a2ddd1b","Type":"ContainerStarted","Data":"52ece0171dd6e9277004d07bfa6f90117e02a8db0560b12a4ab9ab5af01e4d3b"}
Jul 18 13:32:13.845295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.845274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal" event={"ID":"2d0e14a09e5c2e1fbdfe8ed18e98725a","Type":"ContainerStarted","Data":"b0b0aa10764e27043164abda1c4e4e44bb74b727b117b8dc4d909fb493d9986e"}
Jul 18 13:32:13.874915 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.874897    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:13.981106 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:13.981083    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:14.685506 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.685476    2579 apiserver.go:52] "Watching apiserver"
Jul 18 13:32:14.692686 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.692659    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 13:32:14.694256 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.694230    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-j22b9","openshift-cluster-node-tuning-operator/tuned-vrw55","openshift-multus/multus-mrvt5","openshift-multus/network-metrics-daemon-t7vxw","openshift-network-operator/iptables-alerter-dd4hz","openshift-ovn-kubernetes/ovnkube-node-j98mf","kube-system/global-pull-secret-syncer-tk9z7","kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw","openshift-dns/node-resolver-6fjz4","openshift-image-registry/node-ca-f6fff","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal","openshift-multus/multus-additional-cni-plugins-6vj46","openshift-network-diagnostics/network-check-target-npgp2"]
Jul 18 13:32:14.696860 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.696813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.696961 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.696905    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:14.699813 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.699774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.702240 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.702218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.702916 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.702897    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.703191 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.703168    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 13:32:14.703303 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.703204    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jxj27\""
Jul 18 13:32:14.703422 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.703403    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 13:32:14.703508 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.703492    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.703597 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.703582    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 13:32:14.704621 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.704600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:14.704701 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.704688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 13:32:14.704796 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.704701    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:14.704796 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.704713    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2l6r4\""
Jul 18 13:32:14.706424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.706406    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 13:32:14.707088 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.707066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.709612 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.709385    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.709612 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.709556    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.709612 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.709581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vjh78\""
Jul 18 13:32:14.709831 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.709669    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 13:32:14.709831 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.709792    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.712090 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.712073    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.712221 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.712205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.712527 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.712504    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tbnc6\""
Jul 18 13:32:14.712966 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.712795    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.713142 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.713108    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 13:32:14.713288 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.713255    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 13:32:14.713288 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.713276    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 13:32:14.713442 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.713321    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 13:32:14.714696 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.714677    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.714872 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.714856    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lhpm5\""
Jul 18 13:32:14.715125 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715103    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.715125 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715122    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.715537 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-cni-binary-copy\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.715640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-cni-multus\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.715640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-kubelet\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.715640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/68a4b9dc-6879-40dc-8dc3-b974de431b27-iptables-alerter-script\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.715640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-os-release\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8tsj\" (UniqueName: \"kubernetes.io/projected/e53520f7-2b98-4eb5-8475-08e69062b055-kube-api-access-z8tsj\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-system-cni-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-socket-dir-parent\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715853    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-k8s-cni-cncf-io\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.715898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-multus-certs\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb23571a-abdf-4063-9693-d9e77e7a118b-dbus\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-etc-kubernetes\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.715971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-cnibin\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-cnibin\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb23571a-abdf-4063-9693-d9e77e7a118b-kubelet-config\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.716295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68a4b9dc-6879-40dc-8dc3-b974de431b27-host-slash\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.716664 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-cni-bin\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.716723 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-hostroot\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.716806 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716728    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 13:32:14.716806 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mgkb\" (UniqueName: \"kubernetes.io/projected/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-kube-api-access-5mgkb\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.716905 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:14.716905 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb62r\" (UniqueName: \"kubernetes.io/projected/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-kube-api-access-zb62r\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:14.716905 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716888    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbmr4\" (UniqueName: \"kubernetes.io/projected/68a4b9dc-6879-40dc-8dc3-b974de431b27-kube-api-access-wbmr4\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.717061 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.716943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-daemon-config\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.717061 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-system-cni-dir\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.717346 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-os-release\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.717346 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-netns\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.717346 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-conf-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.717487 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-cni-binary-copy\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.717523 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-cni-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.717984 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.717959    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 13:32:14.719117 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.718631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 13:32:14.719117 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.718660    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vttz2\""
Jul 18 13:32:14.719117 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.718767    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 13:32:14.719318 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.719268    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.721925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.721806    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4c2sp\""
Jul 18 13:32:14.721925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.721853    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.722094 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.721960    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.722507 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.722484    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.724889 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.724869    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.725149 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.725131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.725378 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.725354    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.725720 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.725555    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ms952\""
Jul 18 13:32:14.727581 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.727545    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:14.727581 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.727578    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 13:32:14.727767 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.727610    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:14.727767 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.727658    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 13:32:14.728020 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.728002    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lw8h8\""
Jul 18 13:32:14.728129 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.728002    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 13:32:14.746522 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.746496    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jnv9z"
Jul 18 13:32:14.754654 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.754633    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jnv9z"
Jul 18 13:32:14.807293 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.807272    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 13:32:14.818371 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68a4b9dc-6879-40dc-8dc3-b974de431b27-host-slash\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.818485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-run-ovn-kubernetes\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.818485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-cni-bin\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.818485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-systemd\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.818485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-hostroot\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.818485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818457    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68a4b9dc-6879-40dc-8dc3-b974de431b27-host-slash\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.818485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbmr4\" (UniqueName: \"kubernetes.io/projected/68a4b9dc-6879-40dc-8dc3-b974de431b27-kube-api-access-wbmr4\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-hostroot\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-etc-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-daemon-config\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-system-cni-dir\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-netns\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovnkube-config\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-system-cni-dir\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-kubernetes\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-sys\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-netns\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7d9h\" (UniqueName: \"kubernetes.io/projected/70beebf2-81e7-4e13-98d5-62502fa78245-kube-api-access-s7d9h\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.818777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-cni-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-cni-binary-copy\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-hosts-file\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-ovn\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovn-node-metrics-cert\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2kbh\" (UniqueName: \"kubernetes.io/projected/a60ba2ef-9186-4643-9718-a6217a704b1f-kube-api-access-w2kbh\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-etc-selinux\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-cni-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.818996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-systemd-units\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-os-release\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-system-cni-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-socket-dir-parent\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-os-release\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.819172    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:14.819305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-system-cni-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-k8s-cni-cncf-io\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-socket-dir-parent\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-daemon-config\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.819287    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:32:15.319241336 +0000 UTC m=+2.988691056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-k8s-cni-cncf-io\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-run-netns\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-cni-netd\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-cni-binary-copy\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb23571a-abdf-4063-9693-d9e77e7a118b-dbus\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-etc-kubernetes\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb23571a-abdf-4063-9693-d9e77e7a118b-dbus\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-tmp-dir\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819517    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819518    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgn68\" (UniqueName: \"kubernetes.io/projected/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-kube-api-access-cgn68\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-etc-kubernetes\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9bb4daa8-645b-4953-a367-c120e0d6805c-agent-certs\") pod \"konnectivity-agent-j22b9\" (UID: \"9bb4daa8-645b-4953-a367-c120e0d6805c\") " pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.820206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-run\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/733d86a8-681c-40e8-94bd-1be31f891067-host\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-cnibin\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-cnibin\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819751    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-var-lib-kubelet\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-registration-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e53520f7-2b98-4eb5-8475-08e69062b055-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819853    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-lib-modules\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-cni-bin\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mgkb\" (UniqueName: \"kubernetes.io/projected/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-kube-api-access-5mgkb\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb62r\" (UniqueName: \"kubernetes.io/projected/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-kube-api-access-zb62r\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:14.821081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-cni-bin\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.819975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-device-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-os-release\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-conf-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.820029    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-var-lib-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.820138    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:15.320115147 +0000 UTC m=+2.989564843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-multus-conf-dir\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-os-release\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovnkube-script-lib\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjpr9\" (UniqueName: \"kubernetes.io/projected/51e5c3ea-3f91-4525-986a-fff42a4a58f2-kube-api-access-tjpr9\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-modprobe-d\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysctl-conf\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-cni-binary-copy\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-cni-multus\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-kubelet\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.821906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-cni-multus\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-var-lib-kubelet\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/68a4b9dc-6879-40dc-8dc3-b974de431b27-iptables-alerter-script\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-kubelet\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-slash\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-node-log\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-log-socket\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/733d86a8-681c-40e8-94bd-1be31f891067-serviceca\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqkph\" (UniqueName: \"kubernetes.io/projected/733d86a8-681c-40e8-94bd-1be31f891067-kube-api-access-rqkph\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8tsj\" (UniqueName: \"kubernetes.io/projected/e53520f7-2b98-4eb5-8475-08e69062b055-kube-api-access-z8tsj\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-multus-certs\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysconfig\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysctl-d\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-tuned\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a60ba2ef-9186-4643-9718-a6217a704b1f-tmp\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-host-run-multus-certs\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-socket-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.822782 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e53520f7-2b98-4eb5-8475-08e69062b055-cni-binary-copy\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-env-overrides\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820888    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-systemd\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.820988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9bb4daa8-645b-4953-a367-c120e0d6805c-konnectivity-ca\") pod \"konnectivity-agent-j22b9\" (UID: \"9bb4daa8-645b-4953-a367-c120e0d6805c\") " pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821019    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-host\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-sys-fs\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-cnibin\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-cnibin\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb23571a-abdf-4063-9693-d9e77e7a118b-kubelet-config\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb23571a-abdf-4063-9693-d9e77e7a118b-kubelet-config\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:14.823649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.821611    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/68a4b9dc-6879-40dc-8dc3-b974de431b27-iptables-alerter-script\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.825548 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.825514    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 13:32:14.830323 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.830291    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbmr4\" (UniqueName: \"kubernetes.io/projected/68a4b9dc-6879-40dc-8dc3-b974de431b27-kube-api-access-wbmr4\") pod \"iptables-alerter-dd4hz\" (UID: \"68a4b9dc-6879-40dc-8dc3-b974de431b27\") " pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:14.830423 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.830328    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mgkb\" (UniqueName: \"kubernetes.io/projected/90ac29a0-18f4-4eba-a4f5-ac9f2778f396-kube-api-access-5mgkb\") pod \"multus-mrvt5\" (UID: \"90ac29a0-18f4-4eba-a4f5-ac9f2778f396\") " pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:14.830423 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.830299    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8tsj\" (UniqueName: \"kubernetes.io/projected/e53520f7-2b98-4eb5-8475-08e69062b055-kube-api-access-z8tsj\") pod \"multus-additional-cni-plugins-6vj46\" (UID: \"e53520f7-2b98-4eb5-8475-08e69062b055\") " pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:14.830522 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.830494    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb62r\" (UniqueName: \"kubernetes.io/projected/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-kube-api-access-zb62r\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:14.922163 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovnkube-config\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922163 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-kubernetes\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-sys\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7d9h\" (UniqueName: \"kubernetes.io/projected/70beebf2-81e7-4e13-98d5-62502fa78245-kube-api-access-s7d9h\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.922340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-hosts-file\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.922340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-sys\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922288    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-hosts-file\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.922340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-kubernetes\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-ovn\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovn-node-metrics-cert\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-ovn\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2kbh\" (UniqueName: \"kubernetes.io/projected/a60ba2ef-9186-4643-9718-a6217a704b1f-kube-api-access-w2kbh\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-etc-selinux\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.922575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-systemd-units\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-run-netns\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-cni-netd\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922660    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-systemd-units\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-tmp-dir\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgn68\" (UniqueName: \"kubernetes.io/projected/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-kube-api-access-cgn68\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922754    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9bb4daa8-645b-4953-a367-c120e0d6805c-agent-certs\") pod \"konnectivity-agent-j22b9\" (UID: \"9bb4daa8-645b-4953-a367-c120e0d6805c\") " pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-etc-selinux\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-run\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/733d86a8-681c-40e8-94bd-1be31f891067-host\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-var-lib-kubelet\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.922884 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-registration-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovnkube-config\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-lib-modules\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-var-lib-kubelet\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-run-netns\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.922999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-tmp-dir\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-run\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923045    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/733d86a8-681c-40e8-94bd-1be31f891067-host\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-device-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-registration-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-cni-netd\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-device-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923215    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-lib-modules\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-var-lib-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.923485 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovnkube-script-lib\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923299    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-var-lib-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjpr9\" (UniqueName: \"kubernetes.io/projected/51e5c3ea-3f91-4525-986a-fff42a4a58f2-kube-api-access-tjpr9\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-modprobe-d\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysctl-conf\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-kubelet\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-slash\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-node-log\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-log-socket\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-kubelet\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/733d86a8-681c-40e8-94bd-1be31f891067-serviceca\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqkph\" (UniqueName: \"kubernetes.io/projected/733d86a8-681c-40e8-94bd-1be31f891067-kube-api-access-rqkph\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysconfig\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysctl-conf\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-modprobe-d\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-slash\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-node-log\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.924424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-log-socket\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysconfig\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysctl-d\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-tuned\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a60ba2ef-9186-4643-9718-a6217a704b1f-tmp\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-socket-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovnkube-script-lib\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-sysctl-d\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-env-overrides\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-systemd\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9bb4daa8-645b-4953-a367-c120e0d6805c-konnectivity-ca\") pod \"konnectivity-agent-j22b9\" (UID: \"9bb4daa8-645b-4953-a367-c120e0d6805c\") " pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-host\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-socket-dir\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-systemd\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.923992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-sys-fs\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-host\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/70beebf2-81e7-4e13-98d5-62502fa78245-sys-fs\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.925216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/733d86a8-681c-40e8-94bd-1be31f891067-serviceca\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-run-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-run-ovn-kubernetes\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-cni-bin\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-systemd\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-etc-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-etc-openvswitch\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-run-ovn-kubernetes\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51e5c3ea-3f91-4525-986a-fff42a4a58f2-host-cni-bin\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924321    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51e5c3ea-3f91-4525-986a-fff42a4a58f2-env-overrides\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-systemd\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.924698    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9bb4daa8-645b-4953-a367-c120e0d6805c-konnectivity-ca\") pod \"konnectivity-agent-j22b9\" (UID: \"9bb4daa8-645b-4953-a367-c120e0d6805c\") " pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.926157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.925138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51e5c3ea-3f91-4525-986a-fff42a4a58f2-ovn-node-metrics-cert\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.926947 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.926920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a60ba2ef-9186-4643-9718-a6217a704b1f-etc-tuned\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.927262 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.927228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9bb4daa8-645b-4953-a367-c120e0d6805c-agent-certs\") pod \"konnectivity-agent-j22b9\" (UID: \"9bb4daa8-645b-4953-a367-c120e0d6805c\") " pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:14.927667 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.927648    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a60ba2ef-9186-4643-9718-a6217a704b1f-tmp\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:14.929670 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.929635    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:14.929670 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.929660    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:14.929852 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.929676    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:14.929852 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:14.929752    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:32:15.429734584 +0000 UTC m=+3.099184274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:14.930945 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.930919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7d9h\" (UniqueName: \"kubernetes.io/projected/70beebf2-81e7-4e13-98d5-62502fa78245-kube-api-access-s7d9h\") pod \"aws-ebs-csi-driver-node-8ndcw\" (UID: \"70beebf2-81e7-4e13-98d5-62502fa78245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:14.931882 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.931856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgn68\" (UniqueName: \"kubernetes.io/projected/acde3bc8-b671-4cd9-9b88-886cb18a7a8a-kube-api-access-cgn68\") pod \"node-resolver-6fjz4\" (UID: \"acde3bc8-b671-4cd9-9b88-886cb18a7a8a\") " pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:14.932321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.932298    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqkph\" (UniqueName: \"kubernetes.io/projected/733d86a8-681c-40e8-94bd-1be31f891067-kube-api-access-rqkph\") pod \"node-ca-f6fff\" (UID: \"733d86a8-681c-40e8-94bd-1be31f891067\") " pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:14.932817 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.932798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjpr9\" (UniqueName: \"kubernetes.io/projected/51e5c3ea-3f91-4525-986a-fff42a4a58f2-kube-api-access-tjpr9\") pod \"ovnkube-node-j98mf\" (UID: \"51e5c3ea-3f91-4525-986a-fff42a4a58f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:14.932897 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:14.932839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2kbh\" (UniqueName: \"kubernetes.io/projected/a60ba2ef-9186-4643-9718-a6217a704b1f-kube-api-access-w2kbh\") pod \"tuned-vrw55\" (UID: \"a60ba2ef-9186-4643-9718-a6217a704b1f\") " pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:15.013814 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.013786    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6vj46"
Jul 18 13:32:15.022335 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.022315    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mrvt5"
Jul 18 13:32:15.032132 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.032113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dd4hz"
Jul 18 13:32:15.038202 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.038183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:15.046969 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.046950    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw"
Jul 18 13:32:15.055044 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.055022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:15.063111 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.063087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vrw55"
Jul 18 13:32:15.070886 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.070870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6fjz4"
Jul 18 13:32:15.077699 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.077664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f6fff"
Jul 18 13:32:15.192028 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.191991    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:15.326152 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.326067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:15.326152 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.326115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:15.326352 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.326270    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:15.326352 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.326275    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:15.326352 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.326331    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:16.326313782 +0000 UTC m=+3.995763489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:15.326352 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.326347    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:32:16.326340384 +0000 UTC m=+3.995790069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:15.528161 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.528127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:15.528342 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.528303    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:15.528342 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.528328    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:15.528342 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.528340    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:15.528495 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.528412    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:32:16.528390734 +0000 UTC m=+4.197840436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:15.758527 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.758476    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 13:27:15 +0000 UTC" deadline="2028-04-24 13:28:00.976705326 +0000 UTC"
Jul 18 13:32:15.758527 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.758521    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15503h55m45.218189731s"
Jul 18 13:32:15.841564 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:15.841530    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:15.841744 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:15.841681    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:15.889960 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.889925    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacde3bc8_b671_4cd9_9b88_886cb18a7a8a.slice/crio-4d00d530c4e427f7accdc4db6ac130b4dae09676f77d34564b04cded2410dc99 WatchSource:0}: Error finding container 4d00d530c4e427f7accdc4db6ac130b4dae09676f77d34564b04cded2410dc99: Status 404 returned error can't find the container with id 4d00d530c4e427f7accdc4db6ac130b4dae09676f77d34564b04cded2410dc99
Jul 18 13:32:15.891824 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.891802    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733d86a8_681c_40e8_94bd_1be31f891067.slice/crio-5a08542daaa2139c83280179ebe54142406c75e4c4c0816efabcbf00f29a0d8b WatchSource:0}: Error finding container 5a08542daaa2139c83280179ebe54142406c75e4c4c0816efabcbf00f29a0d8b: Status 404 returned error can't find the container with id 5a08542daaa2139c83280179ebe54142406c75e4c4c0816efabcbf00f29a0d8b
Jul 18 13:32:15.895420 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.895330    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60ba2ef_9186_4643_9718_a6217a704b1f.slice/crio-f9263179db287751a95923a08c11c410941c89a79d83885aaff536730f582144 WatchSource:0}: Error finding container f9263179db287751a95923a08c11c410941c89a79d83885aaff536730f582144: Status 404 returned error can't find the container with id f9263179db287751a95923a08c11c410941c89a79d83885aaff536730f582144
Jul 18 13:32:15.895814 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.895789    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53520f7_2b98_4eb5_8475_08e69062b055.slice/crio-1f6cbe67a83129a297b950f786ae3f0760316d21107c77ac11452b4ff02470cb WatchSource:0}: Error finding container 1f6cbe67a83129a297b950f786ae3f0760316d21107c77ac11452b4ff02470cb: Status 404 returned error can't find the container with id 1f6cbe67a83129a297b950f786ae3f0760316d21107c77ac11452b4ff02470cb
Jul 18 13:32:15.897424 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.897357    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a4b9dc_6879_40dc_8dc3_b974de431b27.slice/crio-87e09a8180285383429a0701ef755546f7a052cff72bc02970b36268048b6529 WatchSource:0}: Error finding container 87e09a8180285383429a0701ef755546f7a052cff72bc02970b36268048b6529: Status 404 returned error can't find the container with id 87e09a8180285383429a0701ef755546f7a052cff72bc02970b36268048b6529
Jul 18 13:32:15.899685 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.899639    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70beebf2_81e7_4e13_98d5_62502fa78245.slice/crio-b700478f5ee66cabb50940907d246bb09a1dfeed1d197ede1d620a2e625ccc78 WatchSource:0}: Error finding container b700478f5ee66cabb50940907d246bb09a1dfeed1d197ede1d620a2e625ccc78: Status 404 returned error can't find the container with id b700478f5ee66cabb50940907d246bb09a1dfeed1d197ede1d620a2e625ccc78
Jul 18 13:32:15.900742 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.900664    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ac29a0_18f4_4eba_a4f5_ac9f2778f396.slice/crio-1025ecbc089c9d0ad05e314c1400ec74c54a0f75ffe19f4d5671e877a41c3952 WatchSource:0}: Error finding container 1025ecbc089c9d0ad05e314c1400ec74c54a0f75ffe19f4d5671e877a41c3952: Status 404 returned error can't find the container with id 1025ecbc089c9d0ad05e314c1400ec74c54a0f75ffe19f4d5671e877a41c3952
Jul 18 13:32:15.901596 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:15.901572    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb4daa8_645b_4953_a367_c120e0d6805c.slice/crio-1824ae285cb362f70b9d469ae0f192c05898195084c9fe8bdc04014d12dd4d53 WatchSource:0}: Error finding container 1824ae285cb362f70b9d469ae0f192c05898195084c9fe8bdc04014d12dd4d53: Status 404 returned error can't find the container with id 1824ae285cb362f70b9d469ae0f192c05898195084c9fe8bdc04014d12dd4d53
Jul 18 13:32:16.334246 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.334206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:16.334382 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.334260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:16.334382 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.334349    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:16.334497 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.334381    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:16.334497 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.334411    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:32:18.334396774 +0000 UTC m=+6.003846462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:16.334497 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.334433    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:18.334417431 +0000 UTC m=+6.003867118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:16.537074 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.537036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:16.537305 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.537268    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:16.537305 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.537289    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:16.537305 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.537303    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:16.537495 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.537364    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:32:18.537345235 +0000 UTC m=+6.206794938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:16.758976 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.758873    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 13:27:15 +0000 UTC" deadline="2027-12-28 09:20:32.311726354 +0000 UTC"
Jul 18 13:32:16.758976 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.758941    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12667h48m15.552790846s"
Jul 18 13:32:16.844242 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.844157    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:16.844417 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.844298    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:16.844737 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.844713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:16.844858 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:16.844836    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:16.876110 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.876071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"ab3e1c8fabba9697a7bd77047cb22315e1b7a8006ff843751ae0983b18048bf0"}
Jul 18 13:32:16.885230 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.885198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f6fff" event={"ID":"733d86a8-681c-40e8-94bd-1be31f891067","Type":"ContainerStarted","Data":"5a08542daaa2139c83280179ebe54142406c75e4c4c0816efabcbf00f29a0d8b"}
Jul 18 13:32:16.902456 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.902423    2579 generic.go:356] "Generic (PLEG): container finished" podID="2d0e14a09e5c2e1fbdfe8ed18e98725a" containerID="05999949361d043ccbf5e7f5ff1b5f240d1e06519325e75b19a6480805aea2e5" exitCode=0
Jul 18 13:32:16.902574 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.902524    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal" event={"ID":"2d0e14a09e5c2e1fbdfe8ed18e98725a","Type":"ContainerDied","Data":"05999949361d043ccbf5e7f5ff1b5f240d1e06519325e75b19a6480805aea2e5"}
Jul 18 13:32:16.915899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.915871    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerStarted","Data":"1f6cbe67a83129a297b950f786ae3f0760316d21107c77ac11452b4ff02470cb"}
Jul 18 13:32:16.930882 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.930828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j22b9" event={"ID":"9bb4daa8-645b-4953-a367-c120e0d6805c","Type":"ContainerStarted","Data":"1824ae285cb362f70b9d469ae0f192c05898195084c9fe8bdc04014d12dd4d53"}
Jul 18 13:32:16.941023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.940992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mrvt5" event={"ID":"90ac29a0-18f4-4eba-a4f5-ac9f2778f396","Type":"ContainerStarted","Data":"1025ecbc089c9d0ad05e314c1400ec74c54a0f75ffe19f4d5671e877a41c3952"}
Jul 18 13:32:16.956644 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.956615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw" event={"ID":"70beebf2-81e7-4e13-98d5-62502fa78245","Type":"ContainerStarted","Data":"b700478f5ee66cabb50940907d246bb09a1dfeed1d197ede1d620a2e625ccc78"}
Jul 18 13:32:16.961863 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.961835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dd4hz" event={"ID":"68a4b9dc-6879-40dc-8dc3-b974de431b27","Type":"ContainerStarted","Data":"87e09a8180285383429a0701ef755546f7a052cff72bc02970b36268048b6529"}
Jul 18 13:32:16.967340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.967314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vrw55" event={"ID":"a60ba2ef-9186-4643-9718-a6217a704b1f","Type":"ContainerStarted","Data":"f9263179db287751a95923a08c11c410941c89a79d83885aaff536730f582144"}
Jul 18 13:32:16.991982 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:16.991953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6fjz4" event={"ID":"acde3bc8-b671-4cd9-9b88-886cb18a7a8a","Type":"ContainerStarted","Data":"4d00d530c4e427f7accdc4db6ac130b4dae09676f77d34564b04cded2410dc99"}
Jul 18 13:32:17.014090 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:17.013858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal" event={"ID":"34bcd3343349a9ff2d90b37e5a2ddd1b","Type":"ContainerStarted","Data":"5e9dbafa550c2c5118280f2eb9ed7e61ed514ec59c64ffe1eebb6902ae0003e8"}
Jul 18 13:32:17.841672 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:17.841642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:17.842174 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:17.841800    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:18.063662 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.063536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal" event={"ID":"2d0e14a09e5c2e1fbdfe8ed18e98725a","Type":"ContainerStarted","Data":"f8325d139b4ece723a0fa535cf49fbcf0ee2bb677c06f7d06ff824db86379208"}
Jul 18 13:32:18.080016 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.078862    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-180.ec2.internal" podStartSLOduration=4.078840549 podStartE2EDuration="4.078840549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:18.078573815 +0000 UTC m=+5.748023526" watchObservedRunningTime="2026-07-18 13:32:18.078840549 +0000 UTC m=+5.748290259"
Jul 18 13:32:18.080016 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.079794    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-180.ec2.internal" podStartSLOduration=4.079776905 podStartE2EDuration="4.079776905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:17.030120842 +0000 UTC m=+4.699570552" watchObservedRunningTime="2026-07-18 13:32:18.079776905 +0000 UTC m=+5.749226616"
Jul 18 13:32:18.350656 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.350612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:18.350851 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.350741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:18.350911 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.350872    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:18.350988 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.350935    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:32:22.350916271 +0000 UTC m=+10.020365957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:18.351389 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.351366    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:18.351504 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.351428    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:22.351411161 +0000 UTC m=+10.020860849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:18.552785 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.552745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:18.552969 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.552948    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:18.553051 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.552981    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:18.553051 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.552995    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:18.553154 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.553056    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:32:22.553037051 +0000 UTC m=+10.222486739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:18.841894 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.841863    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:18.842402 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.841993    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:18.842557 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:18.842499    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:18.842655 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:18.842607    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:19.841687 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:19.841648    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:19.841887 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:19.841831    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:20.842155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:20.841593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:20.842155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:20.841658    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:20.842155 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:20.841777    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:20.842155 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:20.841936    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:21.841383 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:21.841345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:21.841604 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:21.841506    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:22.384371 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:22.383868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:22.384371 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:22.383959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:22.384371 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.384081    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:22.384371 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.384141    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:32:30.384121435 +0000 UTC m=+18.053571127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:22.384371 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.384276    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:22.384371 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.384336    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:30.384316957 +0000 UTC m=+18.053766649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:22.585919 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:22.585835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:22.586091 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.586044    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:22.586091 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.586073    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:22.586091 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.586088    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:22.586277 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.586151    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:32:30.586132982 +0000 UTC m=+18.255582671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:22.842494 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:22.842454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:22.842685 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.842589    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:22.843042 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:22.843002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:22.843201 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:22.843141    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:23.841135 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:23.841102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:23.841592 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:23.841250    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:24.841285 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:24.841146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:24.841285 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:24.841202    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:24.841285 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:24.841283    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:24.841839 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:24.841414    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:25.841943 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:25.841697    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:25.841943 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:25.841893    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:26.841324 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:26.841296    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:26.841469 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:26.841401    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:26.841469 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:26.841455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:26.841577 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:26.841541    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:27.841775 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:27.841739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:27.842248 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:27.841886    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:28.841757 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:28.841721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:28.841927 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:28.841842    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:28.841927 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:28.841879    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:28.842299 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:28.841955    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:29.841355 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:29.841323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:29.841564 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:29.841444    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:30.446291 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:30.446252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:30.446753 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:30.446317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:30.446753 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.446403    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:30.446753 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.446448    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:30.446753 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.446464    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:32:46.446448736 +0000 UTC m=+34.115898418 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:30.446753 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.446498    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:46.446480433 +0000 UTC m=+34.115930131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:30.647732 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:30.647699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:30.647879 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.647855    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:30.647879 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.647874    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:30.647957 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.647883    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:30.647957 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.647926    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:32:46.647914349 +0000 UTC m=+34.317364032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:30.842136 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:30.842097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:30.842310 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.842202    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:30.842310 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:30.842251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:30.842430 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:30.842383    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:31.841833 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:31.841799    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:31.842270 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:31.841934    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:32.842170 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:32.842135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:32.842615 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:32.842242    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:32.842615 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:32.842321    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:32.842615 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:32.842431    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:33.841383 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:33.841351    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:33.841551 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:33.841468    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:34.841794 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:34.841763    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:34.842363 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:34.841886    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:34.842363 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:34.841941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:34.842363 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:34.842029    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:35.101270 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.100962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerStarted","Data":"33570718f4aca5c9334c681394f2798d57e8148b6d3ae21eb1126bfd5fd0258e"}
Jul 18 13:32:35.103115 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.103081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j22b9" event={"ID":"9bb4daa8-645b-4953-a367-c120e0d6805c","Type":"ContainerStarted","Data":"a855d86ac7a8446958f4cd8cc815c86c944bcecf814570fb3c0cc3d09c276be9"}
Jul 18 13:32:35.104938 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.104910    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mrvt5" event={"ID":"90ac29a0-18f4-4eba-a4f5-ac9f2778f396","Type":"ContainerStarted","Data":"06906c1b175e5c242dd1a2ee98eebe6cbb94540f68777428f604b81e833c1c68"}
Jul 18 13:32:35.107698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.107652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw" event={"ID":"70beebf2-81e7-4e13-98d5-62502fa78245","Type":"ContainerStarted","Data":"c379983fb147286e9b9c6e0fe99d71f220edef64e7d548a24f8a542071e32c6f"}
Jul 18 13:32:35.110567 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.110524    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vrw55" event={"ID":"a60ba2ef-9186-4643-9718-a6217a704b1f","Type":"ContainerStarted","Data":"e09decb1d328a5bad9c0e4b711a0fc001bd2a6f9500305872c1f0f3187f39951"}
Jul 18 13:32:35.113100 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.113051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6fjz4" event={"ID":"acde3bc8-b671-4cd9-9b88-886cb18a7a8a","Type":"ContainerStarted","Data":"ed272b682cf0714c74a3125dacac24a30e5ce71089fdea321a3db31cdb15919b"}
Jul 18 13:32:35.115274 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.115247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"b54c5117bc402844cc21975c027ae473054eeeb2e50e796c639d4b0b8339baa4"}
Jul 18 13:32:35.117723 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.117697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f6fff" event={"ID":"733d86a8-681c-40e8-94bd-1be31f891067","Type":"ContainerStarted","Data":"b023dc1cb8ad944fb335e7672e30a65661c3a489d3da5c16d8fe992e0e10c239"}
Jul 18 13:32:35.138773 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.138662    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vrw55" podStartSLOduration=3.139221892 podStartE2EDuration="22.138642697s" totalImagesPullingTime="18.999420805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:35.138318169 +0000 UTC m=+22.807767874" watchObservedRunningTime="2026-07-18 13:32:35.138642697 +0000 UTC m=+22.808092403"
Jul 18 13:32:35.154798 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.154464    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mrvt5" podStartSLOduration=3.099415659 podStartE2EDuration="22.15443303s" totalImagesPullingTime="19.055017371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:35.154110278 +0000 UTC m=+22.823559983" watchObservedRunningTime="2026-07-18 13:32:35.15443303 +0000 UTC m=+22.823882733"
Jul 18 13:32:35.169343 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.169301    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j22b9" podStartSLOduration=7.502012137 podStartE2EDuration="22.169289324s" totalImagesPullingTime="14.667277187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:35.169054099 +0000 UTC m=+22.838503805" watchObservedRunningTime="2026-07-18 13:32:35.169289324 +0000 UTC m=+22.838739029"
Jul 18 13:32:35.190554 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.190514    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6fjz4" podStartSLOduration=3.213427898 podStartE2EDuration="22.1905007s" totalImagesPullingTime="18.977072802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:35.190348179 +0000 UTC m=+22.859797887" watchObservedRunningTime="2026-07-18 13:32:35.1905007 +0000 UTC m=+22.859950405"
Jul 18 13:32:35.841592 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:35.841547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:35.841856 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:35.841700    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:36.120988 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.120912    2579 generic.go:356] "Generic (PLEG): container finished" podID="e53520f7-2b98-4eb5-8475-08e69062b055" containerID="33570718f4aca5c9334c681394f2798d57e8148b6d3ae21eb1126bfd5fd0258e" exitCode=0
Jul 18 13:32:36.121171 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.120997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerDied","Data":"33570718f4aca5c9334c681394f2798d57e8148b6d3ae21eb1126bfd5fd0258e"}
Jul 18 13:32:36.124069 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.124037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"bd7b7c82cd58fb3cd30aef17d78b10211b394826f4be244aea469be0c57ba8cc"}
Jul 18 13:32:36.124179 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.124120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"d9d1c441078b5055ea8649e75debe3c4829aee55cda3b70cd46e57b9941bb977"}
Jul 18 13:32:36.124179 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.124172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"7c9694fcd1c582282209215e7640c6685dec60fe70a95776b474b0afb06991a0"}
Jul 18 13:32:36.124293 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.124213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"54cd7ed3b880514921b865b708d64429f69cd438a0b8c0dab03b3c0e96be3195"}
Jul 18 13:32:36.124293 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.124252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"189d68230808d604aaa6a6b67cf7460255a8ea78e2f2396a18480b0609fb5387"}
Jul 18 13:32:36.142160 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.142120    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f6fff" podStartSLOduration=4.141641609 podStartE2EDuration="23.142108617s" totalImagesPullingTime="19.000467008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:35.203783691 +0000 UTC m=+22.873233397" watchObservedRunningTime="2026-07-18 13:32:36.142108617 +0000 UTC m=+23.811558322"
Jul 18 13:32:36.541165 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.541143    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 13:32:36.775061 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.774979    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T13:32:36.541161047Z","UUID":"9836fd32-6327-4522-b657-d58c99bde819","Handler":null,"Name":"","Endpoint":""}
Jul 18 13:32:36.776330 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.776313    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 13:32:36.776439 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.776345    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 13:32:36.841825 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.841801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:36.841921 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:36.841836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:36.841921 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:36.841910    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:36.842204 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:36.842027    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:37.127299 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:37.127220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw" event={"ID":"70beebf2-81e7-4e13-98d5-62502fa78245","Type":"ContainerStarted","Data":"70471c4229cbb2fa276433e46e0bc8bf5d141fbf3603d725f8d745ef247b0914"}
Jul 18 13:32:37.128570 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:37.128546    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dd4hz" event={"ID":"68a4b9dc-6879-40dc-8dc3-b974de431b27","Type":"ContainerStarted","Data":"cd89f65869f7685f46e722de7e2f98fd37ddbe4051de2d22e64a333e97e08d7d"}
Jul 18 13:32:37.141751 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:37.141698    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dd4hz" podStartSLOduration=5.147235955 podStartE2EDuration="24.141660243s" totalImagesPullingTime="18.994424288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:37.141069492 +0000 UTC m=+24.810519196" watchObservedRunningTime="2026-07-18 13:32:37.141660243 +0000 UTC m=+24.811109952"
Jul 18 13:32:37.841987 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:37.841954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:37.842408 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:37.842075    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:38.136598 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:38.136243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"170b5f33e19859a3cc2341514775a6e4a7aa9c3e4cb53b631ce201720b82df22"}
Jul 18 13:32:38.841183 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:38.841144    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:38.841365 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:38.841281    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:38.841365 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:38.841329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:38.841462 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:38.841432    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:39.140922 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:39.140842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw" event={"ID":"70beebf2-81e7-4e13-98d5-62502fa78245","Type":"ContainerStarted","Data":"c830102fa95ec08814a7df4073b89923547badb1bed1b438c2fea5b9a92b01fb"}
Jul 18 13:32:39.157529 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:39.157476    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ndcw" podStartSLOduration=4.362174255 podStartE2EDuration="26.157462339s" totalImagesPullingTime="21.795288084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:39.157188219 +0000 UTC m=+26.826637927" watchObservedRunningTime="2026-07-18 13:32:39.157462339 +0000 UTC m=+26.826912043"
Jul 18 13:32:39.744764 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:39.744727    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:39.745376 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:39.745354    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:39.841436 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:39.841400    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:39.841584 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:39.841527    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:40.571112 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:40.571083    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:40.571833 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:40.571806    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j22b9"
Jul 18 13:32:40.841334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:40.841248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:40.841334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:40.841287    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:40.841543 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:40.841372    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:40.841543 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:40.841495    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:41.841737 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:41.841538    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:41.842376 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:41.841810    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:42.150361 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.150283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" event={"ID":"51e5c3ea-3f91-4525-986a-fff42a4a58f2","Type":"ContainerStarted","Data":"2b11eb06bf17c0064d16e851bd1a721e383e3455cc2b6f8bf41582f8da2bf482"}
Jul 18 13:32:42.150570 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.150548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:42.150664 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.150617    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:42.152192 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.152169    2579 generic.go:356] "Generic (PLEG): container finished" podID="e53520f7-2b98-4eb5-8475-08e69062b055" containerID="2fdc3086314bec76eff4a1c15189d0251ab29146a2b88cf82b57138fc7af318c" exitCode=0
Jul 18 13:32:42.152291 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.152204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerDied","Data":"2fdc3086314bec76eff4a1c15189d0251ab29146a2b88cf82b57138fc7af318c"}
Jul 18 13:32:42.166025 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.165998    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:42.174931 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.174893    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf" podStartSLOduration=10.112921518 podStartE2EDuration="29.174880525s" totalImagesPullingTime="19.061959007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:42.173772252 +0000 UTC m=+29.843221956" watchObservedRunningTime="2026-07-18 13:32:42.174880525 +0000 UTC m=+29.844330211"
Jul 18 13:32:42.844945 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.844917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:42.845350 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:42.845044    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:42.845350 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:42.845094    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:42.845350 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:42.845204    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:43.084179 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.083188    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-npgp2"]
Jul 18 13:32:43.086069 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.085830    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t7vxw"]
Jul 18 13:32:43.086069 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.085902    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tk9z7"]
Jul 18 13:32:43.086069 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.086018    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:43.086266 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:43.086172    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:43.153952 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.153896    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:43.154058 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:43.153990    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:43.154058 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.153895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:43.154452 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:43.154410    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:43.154452 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.154444    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:43.169384 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:43.169367    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:32:44.158515 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:44.158297    2579 generic.go:356] "Generic (PLEG): container finished" podID="e53520f7-2b98-4eb5-8475-08e69062b055" containerID="a086432f89788f4b856427de2f94cc2146f0cec49230f837ccd9857d7490c30b" exitCode=0
Jul 18 13:32:44.158515 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:44.158382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerDied","Data":"a086432f89788f4b856427de2f94cc2146f0cec49230f837ccd9857d7490c30b"}
Jul 18 13:32:44.841356 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:44.841322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:44.841574 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:44.841364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:44.841574 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:44.841329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:44.841574 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:44.841445    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:44.841574 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:44.841522    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:44.841816 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:44.841614    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:46.164907 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.164872    2579 generic.go:356] "Generic (PLEG): container finished" podID="e53520f7-2b98-4eb5-8475-08e69062b055" containerID="2f87de13f0cf592932f71cbe0c5d66f7de71d5672efe3873f995c6cf99386884" exitCode=0
Jul 18 13:32:46.165425 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.164929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerDied","Data":"2f87de13f0cf592932f71cbe0c5d66f7de71d5672efe3873f995c6cf99386884"}
Jul 18 13:32:46.466301 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.466227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:46.466301 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.466281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:46.466470 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.466356    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:46.466470 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.466362    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:46.466470 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.466412    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret podName:cb23571a-abdf-4063-9693-d9e77e7a118b nodeName:}" failed. No retries permitted until 2026-07-18 13:33:18.466397432 +0000 UTC m=+66.135847134 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret") pod "global-pull-secret-syncer-tk9z7" (UID: "cb23571a-abdf-4063-9693-d9e77e7a118b") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:46.466470 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.466429    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:18.466421615 +0000 UTC m=+66.135871299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:46.668447 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.668417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:46.668586 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.668539    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:46.668586 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.668556    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:46.668586 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.668565    2579 projected.go:196] Error preparing data for projected volume kube-api-access-smks9 for pod openshift-network-diagnostics/network-check-target-npgp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:46.668700 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.668610    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9 podName:5cbdec67-c3e3-4b65-863b-735f68685ddf nodeName:}" failed. No retries permitted until 2026-07-18 13:33:18.668594556 +0000 UTC m=+66.338044258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-smks9" (UniqueName: "kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9") pod "network-check-target-npgp2" (UID: "5cbdec67-c3e3-4b65-863b-735f68685ddf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:46.841632 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.841605    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:46.841814 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.841608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:46.841814 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.841717    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk9z7" podUID="cb23571a-abdf-4063-9693-d9e77e7a118b"
Jul 18 13:32:46.841886 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.841837    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:32:46.841919 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:46.841898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:46.841994 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:46.841977    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-npgp2" podUID="5cbdec67-c3e3-4b65-863b-735f68685ddf"
Jul 18 13:32:47.668071 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.667997    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-180.ec2.internal" event="NodeReady"
Jul 18 13:32:47.668498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.668142    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 13:32:47.707723 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.707692    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-867dc7dc44-rj4x2"]
Jul 18 13:32:47.711491 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.711467    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"]
Jul 18 13:32:47.711620 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.711588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.714304 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.714141    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 18 13:32:47.714304 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.714201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xfjwq\""
Jul 18 13:32:47.714483 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.714313    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 18 13:32:47.714548 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.714535    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 18 13:32:47.714944 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.714909    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:47.717318 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.717249    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 18 13:32:47.717318 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.717262    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 18 13:32:47.717853 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.717832    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-67xht\""
Jul 18 13:32:47.722023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.722005    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4nzcb"]
Jul 18 13:32:47.722732 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.722712    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 18 13:32:47.726165 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.726145    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fhc8x"]
Jul 18 13:32:47.726837 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.726468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:47.728890 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.728830    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v6fm2\""
Jul 18 13:32:47.730228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.729390    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 13:32:47.730228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.729611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"]
Jul 18 13:32:47.730228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.729641    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-867dc7dc44-rj4x2"]
Jul 18 13:32:47.730228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.729753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.730228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.730024    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 13:32:47.730228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.730177    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 13:32:47.731967 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.731950    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 13:32:47.732161 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.732142    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 13:32:47.732251 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.732205    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t94bf\""
Jul 18 13:32:47.737180 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.737161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4nzcb"]
Jul 18 13:32:47.740403 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.740376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fhc8x"]
Jul 18 13:32:47.776334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5711ee18-b68f-457a-a167-7b87476eed22-ca-trust-extracted\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776445 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776445 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a1d26ef-9167-4ae1-9279-fd20187cf877-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:47.776445 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s98b\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-kube-api-access-5s98b\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776445 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:47.776634 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-image-registry-private-configuration\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776634 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-bound-sa-token\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776634 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-trusted-ca\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776832 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-registry-certificates\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.776832 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.776689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-installation-pull-secrets\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.862868 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.862832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:47.877721 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-trusted-ca\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.877815 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-registry-certificates\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.877815 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877755    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-installation-pull-secrets\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.877934 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5711ee18-b68f-457a-a167-7b87476eed22-ca-trust-extracted\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.877934 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.878023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.878023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a1d26ef-9167-4ae1-9279-fd20187cf877-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:47.878023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.877993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4w8n\" (UniqueName: \"kubernetes.io/projected/a9bc7408-0f67-49c3-811b-44dac65c9016-kube-api-access-t4w8n\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16a06c2-6872-4981-bf66-8d9266d0ff44-cert\") pod \"ingress-canary-4nzcb\" (UID: \"c16a06c2-6872-4981-bf66-8d9266d0ff44\") " pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h8bh\" (UniqueName: \"kubernetes.io/projected/c16a06c2-6872-4981-bf66-8d9266d0ff44-kube-api-access-7h8bh\") pod \"ingress-canary-4nzcb\" (UID: \"c16a06c2-6872-4981-bf66-8d9266d0ff44\") " pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s98b\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-kube-api-access-5s98b\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.878080    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.878102    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:47.878157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878125    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9bc7408-0f67-49c3-811b-44dac65c9016-config-volume\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.878168    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:48.378148432 +0000 UTC m=+36.047598125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9bc7408-0f67-49c3-811b-44dac65c9016-tmp-dir\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-image-registry-private-configuration\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-bound-sa-token\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.878382    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.878437    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:48.378422591 +0000 UTC m=+36.047872283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:32:47.878477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-registry-certificates\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.878845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878545    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a1d26ef-9167-4ae1-9279-fd20187cf877-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:47.878845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5711ee18-b68f-457a-a167-7b87476eed22-ca-trust-extracted\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.878930 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.878874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-trusted-ca\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.881922 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.881899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-installation-pull-secrets\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.882006 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.881899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-image-registry-private-configuration\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.885848 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.885826    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-bound-sa-token\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.886101 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.886080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s98b\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-kube-api-access-5s98b\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:47.947042 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.946964    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:47.979542 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.979517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.979662 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.979561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4w8n\" (UniqueName: \"kubernetes.io/projected/a9bc7408-0f67-49c3-811b-44dac65c9016-kube-api-access-t4w8n\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.979662 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.979582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16a06c2-6872-4981-bf66-8d9266d0ff44-cert\") pod \"ingress-canary-4nzcb\" (UID: \"c16a06c2-6872-4981-bf66-8d9266d0ff44\") " pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:47.979662 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.979603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h8bh\" (UniqueName: \"kubernetes.io/projected/c16a06c2-6872-4981-bf66-8d9266d0ff44-kube-api-access-7h8bh\") pod \"ingress-canary-4nzcb\" (UID: \"c16a06c2-6872-4981-bf66-8d9266d0ff44\") " pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:47.979662 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.979638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9bc7408-0f67-49c3-811b-44dac65c9016-config-volume\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.979662 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.979640    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:32:47.979935 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:47.979739    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:48.479720606 +0000 UTC m=+36.149170294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:32:47.979935 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.979856    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9bc7408-0f67-49c3-811b-44dac65c9016-tmp-dir\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.980146 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.980128    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9bc7408-0f67-49c3-811b-44dac65c9016-tmp-dir\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.980254 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.980232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9bc7408-0f67-49c3-811b-44dac65c9016-config-volume\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.983232 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.983162    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16a06c2-6872-4981-bf66-8d9266d0ff44-cert\") pod \"ingress-canary-4nzcb\" (UID: \"c16a06c2-6872-4981-bf66-8d9266d0ff44\") " pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:47.988110 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.988087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4w8n\" (UniqueName: \"kubernetes.io/projected/a9bc7408-0f67-49c3-811b-44dac65c9016-kube-api-access-t4w8n\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:47.988320 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:47.988300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h8bh\" (UniqueName: \"kubernetes.io/projected/c16a06c2-6872-4981-bf66-8d9266d0ff44-kube-api-access-7h8bh\") pod \"ingress-canary-4nzcb\" (UID: \"c16a06c2-6872-4981-bf66-8d9266d0ff44\") " pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:48.044459 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.044433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4nzcb"
Jul 18 13:32:48.191176 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.190793    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4nzcb"]
Jul 18 13:32:48.194001 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:32:48.193960    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc16a06c2_6872_4981_bf66_8d9266d0ff44.slice/crio-db0d5e22770a51e984c065e0840abf5b43d4dca20c774df8b4d49e2b75d4eef5 WatchSource:0}: Error finding container db0d5e22770a51e984c065e0840abf5b43d4dca20c774df8b4d49e2b75d4eef5: Status 404 returned error can't find the container with id db0d5e22770a51e984c065e0840abf5b43d4dca20c774df8b4d49e2b75d4eef5
Jul 18 13:32:48.383294 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.383267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:48.383464 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.383304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:48.383464 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.383416    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:32:48.383464 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.383434    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:32:48.383604 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.383488    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:32:48.383604 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.383515    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:49.383490238 +0000 UTC m=+37.052939941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:32:48.383604 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.383548    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:49.383531405 +0000 UTC m=+37.052981093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:32:48.484378 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.484351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:48.484552 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.484500    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:32:48.484606 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:48.484557    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:49.484544595 +0000 UTC m=+37.153994278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:32:48.841099 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.841023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:32:48.841605 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.841023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:32:48.841605 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.841137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:32:48.843569 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.843547    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 13:32:48.843569 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.843563    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 13:32:48.843746 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.843573    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfhtr\""
Jul 18 13:32:48.843746 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.843560    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 13:32:48.843746 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.843698    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l5j6b\""
Jul 18 13:32:48.843906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:48.843699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 13:32:49.173106 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:49.173028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4nzcb" event={"ID":"c16a06c2-6872-4981-bf66-8d9266d0ff44","Type":"ContainerStarted","Data":"db0d5e22770a51e984c065e0840abf5b43d4dca20c774df8b4d49e2b75d4eef5"}
Jul 18 13:32:49.391543 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:49.391494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:49.391754 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:49.391566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:49.391754 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.391657    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:32:49.391754 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.391689    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:32:49.391754 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.391754    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:51.391736903 +0000 UTC m=+39.061186609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:32:49.391754 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.391756    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:32:49.391998 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.391793    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:51.391784015 +0000 UTC m=+39.061233701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:32:49.492747 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:49.492714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:49.492907 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.492883    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:32:49.492982 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:49.492954    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:51.492936858 +0000 UTC m=+39.162386541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:32:50.886105 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:50.886063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:50.934312 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:50.934273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:51.409002 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:51.408961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:51.409189 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:51.409023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:51.409189 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.409136    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:32:51.409189 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.409160    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:32:51.409349 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.409226    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:55.409210986 +0000 UTC m=+43.078660674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:32:51.409349 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.409160    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:32:51.409349 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.409284    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:55.409270195 +0000 UTC m=+43.078719883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:32:51.509909 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:51.509876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:51.510051 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.510032    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:32:51.510101 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:51.510092    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:55.510076847 +0000 UTC m=+43.179526531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:32:52.182205 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:52.182008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4nzcb" event={"ID":"c16a06c2-6872-4981-bf66-8d9266d0ff44","Type":"ContainerStarted","Data":"c6c58373bacbd60882d8677096a21040a13ce986248c22d0ccbd51a1bd0fe7f1"}
Jul 18 13:32:52.184750 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:52.184723    2579 generic.go:356] "Generic (PLEG): container finished" podID="e53520f7-2b98-4eb5-8475-08e69062b055" containerID="46a4e0b494b98e28f7b3e3aa7c4654186a34bc3d12e0dcf0c9499161c4732b78" exitCode=0
Jul 18 13:32:52.184872 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:52.184763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerDied","Data":"46a4e0b494b98e28f7b3e3aa7c4654186a34bc3d12e0dcf0c9499161c4732b78"}
Jul 18 13:32:52.229481 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:52.229375    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4nzcb" podStartSLOduration=1.58669612 podStartE2EDuration="5.229354107s" totalImagesPullingTime="3.642657987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:52.197613852 +0000 UTC m=+39.867063558" watchObservedRunningTime="2026-07-18 13:32:52.229354107 +0000 UTC m=+39.898803812"
Jul 18 13:32:52.593104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:52.593071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:52.693980 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:52.693953    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:53.190006 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:53.189976    2579 generic.go:356] "Generic (PLEG): container finished" podID="e53520f7-2b98-4eb5-8475-08e69062b055" containerID="60156dddbbb0b01e466118a771a7b6e41fe72e03d08fcbab33cc868708862429" exitCode=0
Jul 18 13:32:53.190501 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:53.190054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerDied","Data":"60156dddbbb0b01e466118a771a7b6e41fe72e03d08fcbab33cc868708862429"}
Jul 18 13:32:53.223873 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:53.223844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:53.279999 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:53.279966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:53.328149 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:53.328112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.195464 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:54.195427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vj46" event={"ID":"e53520f7-2b98-4eb5-8475-08e69062b055","Type":"ContainerStarted","Data":"3199a85467f55952107338b61cb0eb2808781560be0f2a4fb589f2a6d0eb3347"}
Jul 18 13:32:54.215998 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:54.215953    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6vj46" podStartSLOduration=13.0470455 podStartE2EDuration="41.215938195s" totalImagesPullingTime="28.168892695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:54.21477342 +0000 UTC m=+41.884223124" watchObservedRunningTime="2026-07-18 13:32:54.215938195 +0000 UTC m=+41.885387900"
Jul 18 13:32:55.438895 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:55.438859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:32:55.439250 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:55.438903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:32:55.439250 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.439012    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:32:55.439250 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.439028    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:32:55.439250 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.439040    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:32:55.439250 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.439086    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:03.439072591 +0000 UTC m=+51.108522278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:32:55.439250 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.439100    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:03.439094646 +0000 UTC m=+51.108544330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:32:55.539554 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:32:55.539521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:32:55.539702 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.539647    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:32:55.539738 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:32:55.539725    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:03.539711916 +0000 UTC m=+51.209161599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:03.496248 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:03.496201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:33:03.496248 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:03.496250    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:33:03.496700 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.496345    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:33:03.496700 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.496361    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:33:03.496700 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.496382    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:33:03.496700 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.496398    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:19.496384026 +0000 UTC m=+67.165833710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:33:03.496700 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.496437    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:19.496423268 +0000 UTC m=+67.165872952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:33:03.596849 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:03.596801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:33:03.597062 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.596902    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:03.597062 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:03.596976    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:19.596959624 +0000 UTC m=+67.266409306 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:15.170589 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:15.170561    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mf"
Jul 18 13:33:18.506057 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.506012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:33:18.506432 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.506109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:33:18.508917 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.508900    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 13:33:18.509170 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.509155    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 13:33:18.516812 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:18.516788    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 13:33:18.516938 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:18.516847    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:34:22.516831203 +0000 UTC m=+130.186280889 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : secret "metrics-daemon-secret" not found
Jul 18 13:33:18.520501 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.520484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb23571a-abdf-4063-9693-d9e77e7a118b-original-pull-secret\") pod \"global-pull-secret-syncer-tk9z7\" (UID: \"cb23571a-abdf-4063-9693-d9e77e7a118b\") " pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:33:18.554543 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.554517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk9z7"
Jul 18 13:33:18.669276 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.669242    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tk9z7"]
Jul 18 13:33:18.672610 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:33:18.672582    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb23571a_abdf_4063_9693_d9e77e7a118b.slice/crio-d4a894cbbc751c5f57513e9e61010fe2a3d847f210c6ac01a0fb90ca2bc1dd6a WatchSource:0}: Error finding container d4a894cbbc751c5f57513e9e61010fe2a3d847f210c6ac01a0fb90ca2bc1dd6a: Status 404 returned error can't find the container with id d4a894cbbc751c5f57513e9e61010fe2a3d847f210c6ac01a0fb90ca2bc1dd6a
Jul 18 13:33:18.707511 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.707483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:33:18.710087 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.710071    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 13:33:18.720023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.720009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 13:33:18.730945 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.730929    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smks9\" (UniqueName: \"kubernetes.io/projected/5cbdec67-c3e3-4b65-863b-735f68685ddf-kube-api-access-smks9\") pod \"network-check-target-npgp2\" (UID: \"5cbdec67-c3e3-4b65-863b-735f68685ddf\") " pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:33:18.864455 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.864383    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfhtr\""
Jul 18 13:33:18.872264 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.872238    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:33:18.985247 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:18.985216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-npgp2"]
Jul 18 13:33:18.987649 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:33:18.987625    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbdec67_c3e3_4b65_863b_735f68685ddf.slice/crio-56460acc449294d48ce0a23ffb2802f9b285a44f09328d1767bc300ae1938633 WatchSource:0}: Error finding container 56460acc449294d48ce0a23ffb2802f9b285a44f09328d1767bc300ae1938633: Status 404 returned error can't find the container with id 56460acc449294d48ce0a23ffb2802f9b285a44f09328d1767bc300ae1938633
Jul 18 13:33:19.253243 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:19.253209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-npgp2" event={"ID":"5cbdec67-c3e3-4b65-863b-735f68685ddf","Type":"ContainerStarted","Data":"56460acc449294d48ce0a23ffb2802f9b285a44f09328d1767bc300ae1938633"}
Jul 18 13:33:19.254167 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:19.254148    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tk9z7" event={"ID":"cb23571a-abdf-4063-9693-d9e77e7a118b","Type":"ContainerStarted","Data":"d4a894cbbc751c5f57513e9e61010fe2a3d847f210c6ac01a0fb90ca2bc1dd6a"}
Jul 18 13:33:19.512487 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:19.512401    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:33:19.512487 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:19.512455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:33:19.512953 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.512540    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:33:19.512953 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.512567    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:33:19.512953 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.512573    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:33:19.512953 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.512636    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:51.512615948 +0000 UTC m=+99.182065655 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:33:19.512953 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.512654    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:51.512645467 +0000 UTC m=+99.182095150 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:33:19.613560 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:19.613527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:33:19.613745 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.613701    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:19.613822 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:19.613774    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:51.613753486 +0000 UTC m=+99.283203172 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:24.268879 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:24.268840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-npgp2" event={"ID":"5cbdec67-c3e3-4b65-863b-735f68685ddf","Type":"ContainerStarted","Data":"09ead56ea883ca69dd791054b3701894e533b4961542646e9ff804cc9995f93a"}
Jul 18 13:33:24.269306 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:24.268966    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:33:24.270258 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:24.270237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tk9z7" event={"ID":"cb23571a-abdf-4063-9693-d9e77e7a118b","Type":"ContainerStarted","Data":"052087a1d3e86be74a11cebfad34a4d32b3b399ab3af7e04efe1ce6a30a27832"}
Jul 18 13:33:24.284115 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:24.284060    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-npgp2" podStartSLOduration=66.582609627 podStartE2EDuration="1m11.284049146s" totalImagesPullingTime="4.701439519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:24.283201125 +0000 UTC m=+71.952650831" watchObservedRunningTime="2026-07-18 13:33:24.284049146 +0000 UTC m=+71.953498850"
Jul 18 13:33:51.539642 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:51.539498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:33:51.539642 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:51.539540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:33:51.540184 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.539644    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:33:51.540184 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.539664    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:33:51.540184 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.539702    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:33:51.540184 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.539714    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:34:55.539699258 +0000 UTC m=+163.209148942 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:33:51.540184 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.539775    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:34:55.539755294 +0000 UTC m=+163.209204979 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:33:51.640487 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:51.640456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:33:51.640621 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.640600    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:51.640693 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:33:51.640665    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:34:55.640644826 +0000 UTC m=+163.310094514 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:55.275009 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:55.274976    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-npgp2"
Jul 18 13:33:55.289853 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:33:55.289798    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tk9z7" podStartSLOduration=97.272747768 podStartE2EDuration="1m42.289785258s" totalImagesPullingTime="5.01703749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:24.2983102 +0000 UTC m=+71.967759906" watchObservedRunningTime="2026-07-18 13:33:55.289785258 +0000 UTC m=+102.959234962"
Jul 18 13:34:05.530267 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.530229    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"]
Jul 18 13:34:05.532183 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.532164    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.534532 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.534501    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 18 13:34:05.534532 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.534522    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 18 13:34:05.534718 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.534555    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 18 13:34:05.535430 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.535418    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 18 13:34:05.542428 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.542405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"]
Jul 18 13:34:05.638143 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.638111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a8f5d7-3dea-4f82-ad70-ed2232747282-tmp\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.638302 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.638148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjqc7\" (UniqueName: \"kubernetes.io/projected/88a8f5d7-3dea-4f82-ad70-ed2232747282-kube-api-access-vjqc7\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.638302 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.638182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/88a8f5d7-3dea-4f82-ad70-ed2232747282-klusterlet-config\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.739120 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.739087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/88a8f5d7-3dea-4f82-ad70-ed2232747282-klusterlet-config\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.739219 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.739174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a8f5d7-3dea-4f82-ad70-ed2232747282-tmp\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.739219 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.739200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqc7\" (UniqueName: \"kubernetes.io/projected/88a8f5d7-3dea-4f82-ad70-ed2232747282-kube-api-access-vjqc7\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.739559 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.739537    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a8f5d7-3dea-4f82-ad70-ed2232747282-tmp\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.741542 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.741520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/88a8f5d7-3dea-4f82-ad70-ed2232747282-klusterlet-config\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.746304 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.746286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqc7\" (UniqueName: \"kubernetes.io/projected/88a8f5d7-3dea-4f82-ad70-ed2232747282-kube-api-access-vjqc7\") pod \"klusterlet-addon-workmgr-76f99d44cd-wd7vg\" (UID: \"88a8f5d7-3dea-4f82-ad70-ed2232747282\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.840795 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.840702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:05.958323 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:05.958289    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"]
Jul 18 13:34:05.962185 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:34:05.962143    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a8f5d7_3dea_4f82_ad70_ed2232747282.slice/crio-21db9ac6329a74856dd6a926c11e0fde6e82b42cb119e2464afe8079141d4f97 WatchSource:0}: Error finding container 21db9ac6329a74856dd6a926c11e0fde6e82b42cb119e2464afe8079141d4f97: Status 404 returned error can't find the container with id 21db9ac6329a74856dd6a926c11e0fde6e82b42cb119e2464afe8079141d4f97
Jul 18 13:34:06.367868 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:06.367837    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg" event={"ID":"88a8f5d7-3dea-4f82-ad70-ed2232747282","Type":"ContainerStarted","Data":"21db9ac6329a74856dd6a926c11e0fde6e82b42cb119e2464afe8079141d4f97"}
Jul 18 13:34:10.379252 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:10.379220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg" event={"ID":"88a8f5d7-3dea-4f82-ad70-ed2232747282","Type":"ContainerStarted","Data":"3f9f69135efd6119479193a3c44d435e46a08c44036f6d46927f01494541dbdd"}
Jul 18 13:34:10.379592 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:10.379415    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:10.380920 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:10.380901    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:34:10.396228 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:10.396187    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg" podStartSLOduration=1.479054865 podStartE2EDuration="5.396174264s" totalImagesPullingTime="3.917119399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:34:10.394689339 +0000 UTC m=+118.064139038" watchObservedRunningTime="2026-07-18 13:34:10.396174264 +0000 UTC m=+118.065623969"
Jul 18 13:34:22.567039 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:22.567004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:34:22.567417 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:22.567109    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 13:34:22.567417 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:22.567168    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:24.567154276 +0000 UTC m=+252.236603960 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : secret "metrics-daemon-secret" not found
Jul 18 13:34:50.724768 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:50.724720    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" podUID="5711ee18-b68f-457a-a167-7b87476eed22"
Jul 18 13:34:50.733845 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:50.733817    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd" podUID="9a1d26ef-9167-4ae1-9279-fd20187cf877"
Jul 18 13:34:50.749081 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:50.749061    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fhc8x" podUID="a9bc7408-0f67-49c3-811b-44dac65c9016"
Jul 18 13:34:51.472746 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:51.472716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:34:51.472746 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:51.472730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:34:51.472939 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:51.472842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:34:51.867665 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:51.867577    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:34:55.585993 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:55.585962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:34:55.586370 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:55.586001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:34:55.586370 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.586106    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 13:34:55.586370 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.586111    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:34:55.586370 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.586129    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-867dc7dc44-rj4x2: secret "image-registry-tls" not found
Jul 18 13:34:55.586370 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.586164    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert podName:9a1d26ef-9167-4ae1-9279-fd20187cf877 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:57.58615191 +0000 UTC m=+285.255601593 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8lpzd" (UID: "9a1d26ef-9167-4ae1-9279-fd20187cf877") : secret "networking-console-plugin-cert" not found
Jul 18 13:34:55.586370 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.586177    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls podName:5711ee18-b68f-457a-a167-7b87476eed22 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:57.586170625 +0000 UTC m=+285.255620307 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls") pod "image-registry-867dc7dc44-rj4x2" (UID: "5711ee18-b68f-457a-a167-7b87476eed22") : secret "image-registry-tls" not found
Jul 18 13:34:55.687123 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:34:55.687090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:34:55.687264 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.687219    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:34:55.687310 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:34:55.687269    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls podName:a9bc7408-0f67-49c3-811b-44dac65c9016 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:57.687256349 +0000 UTC m=+285.356706032 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls") pod "dns-default-fhc8x" (UID: "a9bc7408-0f67-49c3-811b-44dac65c9016") : secret "dns-default-metrics-tls" not found
Jul 18 13:35:03.841318 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:35:03.841284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:36:10.379980 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:10.379933    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg" podUID="88a8f5d7-3dea-4f82-ad70-ed2232747282" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused"
Jul 18 13:36:10.654107 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:10.654035    2579 generic.go:356] "Generic (PLEG): container finished" podID="88a8f5d7-3dea-4f82-ad70-ed2232747282" containerID="3f9f69135efd6119479193a3c44d435e46a08c44036f6d46927f01494541dbdd" exitCode=1
Jul 18 13:36:10.654107 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:10.654072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg" event={"ID":"88a8f5d7-3dea-4f82-ad70-ed2232747282","Type":"ContainerDied","Data":"3f9f69135efd6119479193a3c44d435e46a08c44036f6d46927f01494541dbdd"}
Jul 18 13:36:10.654352 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:10.654340    2579 scope.go:117] "RemoveContainer" containerID="3f9f69135efd6119479193a3c44d435e46a08c44036f6d46927f01494541dbdd"
Jul 18 13:36:11.658156 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:11.658121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg" event={"ID":"88a8f5d7-3dea-4f82-ad70-ed2232747282","Type":"ContainerStarted","Data":"2f4a4c41db0c3a39b31716290b75de46b60994a9215360886a39e2e26bdff65f"}
Jul 18 13:36:11.658520 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:11.658348    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:36:11.658953 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:11.658934    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76f99d44cd-wd7vg"
Jul 18 13:36:22.399298 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.399262    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"]
Jul 18 13:36:22.402206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.402186    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.404413 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.404390    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 18 13:36:22.404565 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.404466    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-n5kr2\""
Jul 18 13:36:22.405341 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.405304    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.406408 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.406391    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.406504 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.406463    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 18 13:36:22.415466 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.415446    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"]
Jul 18 13:36:22.483056 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.483029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.483181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.483062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4a2e8610-58a1-4df8-8f77-d207ad218433-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.483181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.483089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j842v\" (UniqueName: \"kubernetes.io/projected/4a2e8610-58a1-4df8-8f77-d207ad218433-kube-api-access-j842v\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.495640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.495613    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"]
Jul 18 13:36:22.498480 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.498466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:22.500724 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.500703    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.500826 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.500711    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wlt6r\""
Jul 18 13:36:22.500882 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.500831    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 18 13:36:22.500970 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.500952    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.510797 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.510776    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"]
Jul 18 13:36:22.583335 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.583303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvk6h\" (UniqueName: \"kubernetes.io/projected/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-kube-api-access-hvk6h\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:22.583477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.583342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:22.583477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.583367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.583477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.583392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4a2e8610-58a1-4df8-8f77-d207ad218433-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.583477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.583429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j842v\" (UniqueName: \"kubernetes.io/projected/4a2e8610-58a1-4df8-8f77-d207ad218433-kube-api-access-j842v\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.583608 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:22.583485    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:22.583608 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:22.583559    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls podName:4a2e8610-58a1-4df8-8f77-d207ad218433 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:23.083540198 +0000 UTC m=+250.752989890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4584v" (UID: "4a2e8610-58a1-4df8-8f77-d207ad218433") : secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:22.584027 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.584009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4a2e8610-58a1-4df8-8f77-d207ad218433-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.591490 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.591465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j842v\" (UniqueName: \"kubernetes.io/projected/4a2e8610-58a1-4df8-8f77-d207ad218433-kube-api-access-j842v\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:22.683951 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.683880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvk6h\" (UniqueName: \"kubernetes.io/projected/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-kube-api-access-hvk6h\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:22.683951 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.683930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:22.684084 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:22.684003    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 13:36:22.684084 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:22.684048    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls podName:fbbc0ed0-f974-47ed-937a-b7aa8b103b06 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:23.184037162 +0000 UTC m=+250.853486846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-rvd5j" (UID: "fbbc0ed0-f974-47ed-937a-b7aa8b103b06") : secret "samples-operator-tls" not found
Jul 18 13:36:22.693961 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:22.693934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvk6h\" (UniqueName: \"kubernetes.io/projected/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-kube-api-access-hvk6h\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:23.086520 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:23.086472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:23.086704 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:23.086645    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:23.086749 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:23.086727    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls podName:4a2e8610-58a1-4df8-8f77-d207ad218433 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:24.086709334 +0000 UTC m=+251.756159037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4584v" (UID: "4a2e8610-58a1-4df8-8f77-d207ad218433") : secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:23.187265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:23.187228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:23.187414 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:23.187368    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 13:36:23.187452 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:23.187426    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls podName:fbbc0ed0-f974-47ed-937a-b7aa8b103b06 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:24.187412198 +0000 UTC m=+251.856861881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-rvd5j" (UID: "fbbc0ed0-f974-47ed-937a-b7aa8b103b06") : secret "samples-operator-tls" not found
Jul 18 13:36:24.094583 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:24.094548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:24.094950 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:24.094660    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:24.094950 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:24.094728    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls podName:4a2e8610-58a1-4df8-8f77-d207ad218433 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:26.09471585 +0000 UTC m=+253.764165533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4584v" (UID: "4a2e8610-58a1-4df8-8f77-d207ad218433") : secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:24.195459 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:24.195436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:24.195584 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:24.195565    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 13:36:24.195645 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:24.195637    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls podName:fbbc0ed0-f974-47ed-937a-b7aa8b103b06 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:26.195622713 +0000 UTC m=+253.865072395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-rvd5j" (UID: "fbbc0ed0-f974-47ed-937a-b7aa8b103b06") : secret "samples-operator-tls" not found
Jul 18 13:36:24.597333 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:24.597302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:36:24.597489 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:24.597413    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 13:36:24.597489 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:24.597468    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs podName:bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8 nodeName:}" failed. No retries permitted until 2026-07-18 13:38:26.597451788 +0000 UTC m=+374.266901693 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs") pod "network-metrics-daemon-t7vxw" (UID: "bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8") : secret "metrics-daemon-secret" not found
Jul 18 13:36:26.108525 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:26.108477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:26.108925 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:26.108611    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:26.108925 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:26.108684    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls podName:4a2e8610-58a1-4df8-8f77-d207ad218433 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:30.10865769 +0000 UTC m=+257.778107374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4584v" (UID: "4a2e8610-58a1-4df8-8f77-d207ad218433") : secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:26.209362 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:26.209319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:26.209456 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:26.209442    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 13:36:26.209506 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:26.209497    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls podName:fbbc0ed0-f974-47ed-937a-b7aa8b103b06 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:30.209480907 +0000 UTC m=+257.878930590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-rvd5j" (UID: "fbbc0ed0-f974-47ed-937a-b7aa8b103b06") : secret "samples-operator-tls" not found
Jul 18 13:36:27.766759 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.766727    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"]
Jul 18 13:36:27.769812 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.769795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"
Jul 18 13:36:27.772048 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.772022    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 18 13:36:27.772157 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.772025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-csh9q\""
Jul 18 13:36:27.772872 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.772854    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 18 13:36:27.779154 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.779133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"]
Jul 18 13:36:27.819959 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.819937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bln8\" (UniqueName: \"kubernetes.io/projected/64a0b6bb-bb2f-43cf-ac09-0e2babad1c63-kube-api-access-5bln8\") pod \"migrator-7cc9c7ffd7-4lm28\" (UID: \"64a0b6bb-bb2f-43cf-ac09-0e2babad1c63\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"
Jul 18 13:36:27.920725 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.920701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bln8\" (UniqueName: \"kubernetes.io/projected/64a0b6bb-bb2f-43cf-ac09-0e2babad1c63-kube-api-access-5bln8\") pod \"migrator-7cc9c7ffd7-4lm28\" (UID: \"64a0b6bb-bb2f-43cf-ac09-0e2babad1c63\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"
Jul 18 13:36:27.927548 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:27.927526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bln8\" (UniqueName: \"kubernetes.io/projected/64a0b6bb-bb2f-43cf-ac09-0e2babad1c63-kube-api-access-5bln8\") pod \"migrator-7cc9c7ffd7-4lm28\" (UID: \"64a0b6bb-bb2f-43cf-ac09-0e2babad1c63\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"
Jul 18 13:36:28.039824 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.039762    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sjvqs"]
Jul 18 13:36:28.042886 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.042867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.045085 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.045064    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 13:36:28.045282 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.045260    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 13:36:28.045375 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.045266    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 13:36:28.045524 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.045505    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 13:36:28.045592 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.045540    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8pprp\""
Jul 18 13:36:28.057661 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.057639    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sjvqs"]
Jul 18 13:36:28.078894 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.078868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"
Jul 18 13:36:28.121555 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.121490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.121746 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.121639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw2sn\" (UniqueName: \"kubernetes.io/projected/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-api-access-tw2sn\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.121797 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.121787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cc4025b-f7ea-4d85-99f3-c33b38439416-data-volume\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.121831 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.121817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.121871 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.121846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7cc4025b-f7ea-4d85-99f3-c33b38439416-crio-socket\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.193664 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.193636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28"]
Jul 18 13:36:28.196811 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:28.196785    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a0b6bb_bb2f_43cf_ac09_0e2babad1c63.slice/crio-3c5b1b705bf31070b949609d5a5edb86deac4b2c5a21834b3d4f7fd9928fbefb WatchSource:0}: Error finding container 3c5b1b705bf31070b949609d5a5edb86deac4b2c5a21834b3d4f7fd9928fbefb: Status 404 returned error can't find the container with id 3c5b1b705bf31070b949609d5a5edb86deac4b2c5a21834b3d4f7fd9928fbefb
Jul 18 13:36:28.222358 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cc4025b-f7ea-4d85-99f3-c33b38439416-data-volume\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7cc4025b-f7ea-4d85-99f3-c33b38439416-crio-socket\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222545 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222545 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw2sn\" (UniqueName: \"kubernetes.io/projected/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-api-access-tw2sn\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222619 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7cc4025b-f7ea-4d85-99f3-c33b38439416-crio-socket\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222687 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.222619    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.222687 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.222651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cc4025b-f7ea-4d85-99f3-c33b38439416-data-volume\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.222687 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.222683    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:28.722652377 +0000 UTC m=+256.392102063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.222837 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.222691    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.222837 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.222756    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:28.722738709 +0000 UTC m=+256.392188394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.231400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.231384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw2sn\" (UniqueName: \"kubernetes.io/projected/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-api-access-tw2sn\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.699809 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.699770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28" event={"ID":"64a0b6bb-bb2f-43cf-ac09-0e2babad1c63","Type":"ContainerStarted","Data":"3c5b1b705bf31070b949609d5a5edb86deac4b2c5a21834b3d4f7fd9928fbefb"}
Jul 18 13:36:28.727526 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.727496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.727712 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:28.727540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:28.727712 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.727643    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.727712 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.727647    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.727846 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.727720    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:29.727702205 +0000 UTC m=+257.397151888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.727846 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:28.727736    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:29.727730452 +0000 UTC m=+257.397180134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:29.704364 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:29.704325    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28" event={"ID":"64a0b6bb-bb2f-43cf-ac09-0e2babad1c63","Type":"ContainerStarted","Data":"52f78a3405746f04290cb63974b3620f3323448ab34ebc547966911136d54732"}
Jul 18 13:36:29.704753 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:29.704389    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28" event={"ID":"64a0b6bb-bb2f-43cf-ac09-0e2babad1c63","Type":"ContainerStarted","Data":"5ef5a9e7aa85d26541731aea89ffe04b0134d7dc425cc2b9506c4dcdcc2dfd5a"}
Jul 18 13:36:29.720978 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:29.720936    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-4lm28" podStartSLOduration=1.764504392 podStartE2EDuration="2.720922209s" totalImagesPullingTime="956.417817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:29.719690962 +0000 UTC m=+257.389140661" watchObservedRunningTime="2026-07-18 13:36:29.720922209 +0000 UTC m=+257.390371916"
Jul 18 13:36:29.736178 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:29.736156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:29.736288 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:29.736198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:29.736288 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:29.736278    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:29.736354 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:29.736324    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:31.736313094 +0000 UTC m=+259.405762777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:29.736393 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:29.736359    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:29.736428 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:29.736412    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:31.73639961 +0000 UTC m=+259.405849294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:30.139357 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:30.139326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:30.139516 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:30.139446    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:30.139516 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:30.139494    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls podName:4a2e8610-58a1-4df8-8f77-d207ad218433 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:38.139481588 +0000 UTC m=+265.808931271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4584v" (UID: "4a2e8610-58a1-4df8-8f77-d207ad218433") : secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:30.240404 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:30.240376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:30.240542 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:30.240521    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 13:36:30.240593 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:30.240584    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls podName:fbbc0ed0-f974-47ed-937a-b7aa8b103b06 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:38.24057059 +0000 UTC m=+265.910020273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-rvd5j" (UID: "fbbc0ed0-f974-47ed-937a-b7aa8b103b06") : secret "samples-operator-tls" not found
Jul 18 13:36:30.639872 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:30.639849    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6fjz4_acde3bc8-b671-4cd9-9b88-886cb18a7a8a/dns-node-resolver/0.log"
Jul 18 13:36:31.751218 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:31.751187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:31.751551 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:31.751232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:31.751551 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:31.751306    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:31.751551 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:31.751328    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:31.751551 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:31.751352    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:35.751340062 +0000 UTC m=+263.420789744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:31.751551 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:31.751381    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:35.751368007 +0000 UTC m=+263.420817690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:31.845559 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:31.845531    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f6fff_733d86a8-681c-40e8-94bd-1be31f891067/node-ca/0.log"
Jul 18 13:36:32.440347 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:32.440321    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4nzcb_c16a06c2-6872-4981-bf66-8d9266d0ff44/serve-healthcheck-canary/0.log"
Jul 18 13:36:33.040367 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:33.040333    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-4lm28_64a0b6bb-bb2f-43cf-ac09-0e2babad1c63/migrator/0.log"
Jul 18 13:36:33.239970 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:33.239944    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-4lm28_64a0b6bb-bb2f-43cf-ac09-0e2babad1c63/graceful-termination/0.log"
Jul 18 13:36:35.781392 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:35.781314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:35.781392 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:35.781369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:35.781802 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:35.781480    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:35.781802 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:35.781502    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:35.781802 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:35.781543    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:43.781525317 +0000 UTC m=+271.450975006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:35.781802 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:35.781558    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm podName:7cc4025b-f7ea-4d85-99f3-c33b38439416 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:43.781551703 +0000 UTC m=+271.451001386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjvqs" (UID: "7cc4025b-f7ea-4d85-99f3-c33b38439416") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:38.199463 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:38.199430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:38.199865 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:38.199573    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:38.199865 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:38.199630    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls podName:4a2e8610-58a1-4df8-8f77-d207ad218433 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:54.19961318 +0000 UTC m=+281.869062882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4584v" (UID: "4a2e8610-58a1-4df8-8f77-d207ad218433") : secret "cluster-monitoring-operator-tls" not found
Jul 18 13:36:38.300106 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:38.300074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:38.302327 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:38.302309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbbc0ed0-f974-47ed-937a-b7aa8b103b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-rvd5j\" (UID: \"fbbc0ed0-f974-47ed-937a-b7aa8b103b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:38.406950 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:38.406923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"
Jul 18 13:36:38.527958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:38.527927    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j"]
Jul 18 13:36:38.731768 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:38.731742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j" event={"ID":"fbbc0ed0-f974-47ed-937a-b7aa8b103b06","Type":"ContainerStarted","Data":"e3265818b1d285bcbd08ec9ebba7956ae48671e4c0811166d770708e06bdf81b"}
Jul 18 13:36:40.738963 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:40.738933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j" event={"ID":"fbbc0ed0-f974-47ed-937a-b7aa8b103b06","Type":"ContainerStarted","Data":"6144780512a929832ee969d9083198295560265a45e763dd17c70ebf5da6f3c7"}
Jul 18 13:36:40.739321 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:40.738993    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j" event={"ID":"fbbc0ed0-f974-47ed-937a-b7aa8b103b06","Type":"ContainerStarted","Data":"0a7fecc5a3c8db61feebec9c2e234b71568ef2323d4b035006fd5e2a9fda6af1"}
Jul 18 13:36:40.756438 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:40.756394    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-rvd5j" podStartSLOduration=17.076235296 podStartE2EDuration="18.756379323s" totalImagesPullingTime="1.680144027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:40.755116547 +0000 UTC m=+268.424566302" watchObservedRunningTime="2026-07-18 13:36:40.756379323 +0000 UTC m=+268.425829028"
Jul 18 13:36:43.843499 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:43.843465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:43.843888 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:43.843513    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:43.844051 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:43.844033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7cc4025b-f7ea-4d85-99f3-c33b38439416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:43.845705 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:43.845664    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7cc4025b-f7ea-4d85-99f3-c33b38439416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjvqs\" (UID: \"7cc4025b-f7ea-4d85-99f3-c33b38439416\") " pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:43.951584 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:43.951560    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sjvqs"
Jul 18 13:36:44.074405 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:44.074383    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sjvqs"]
Jul 18 13:36:44.076573 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:44.076541    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc4025b_f7ea_4d85_99f3_c33b38439416.slice/crio-342090870a91425da159e71332ef6ec9ab3afcf32f42342ae377fded1832a10c WatchSource:0}: Error finding container 342090870a91425da159e71332ef6ec9ab3afcf32f42342ae377fded1832a10c: Status 404 returned error can't find the container with id 342090870a91425da159e71332ef6ec9ab3afcf32f42342ae377fded1832a10c
Jul 18 13:36:44.753633 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:44.753586    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjvqs" event={"ID":"7cc4025b-f7ea-4d85-99f3-c33b38439416","Type":"ContainerStarted","Data":"342090870a91425da159e71332ef6ec9ab3afcf32f42342ae377fded1832a10c"}
Jul 18 13:36:49.465697 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.464563    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-rd9tj"]
Jul 18 13:36:49.467967 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.467943    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:36:49.470513 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.470485    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4qx7g\""
Jul 18 13:36:49.470811 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.470787    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 18 13:36:49.471001 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.470984    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 18 13:36:49.481829 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.481803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rd9tj"]
Jul 18 13:36:49.590193 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.590136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vmnv\" (UniqueName: \"kubernetes.io/projected/5a062f3a-343a-437a-b521-ae76e8795f27-kube-api-access-5vmnv\") pod \"downloads-5bc67dcd46-rd9tj\" (UID: \"5a062f3a-343a-437a-b521-ae76e8795f27\") " pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:36:49.690880 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.690834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vmnv\" (UniqueName: \"kubernetes.io/projected/5a062f3a-343a-437a-b521-ae76e8795f27-kube-api-access-5vmnv\") pod \"downloads-5bc67dcd46-rd9tj\" (UID: \"5a062f3a-343a-437a-b521-ae76e8795f27\") " pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:36:49.699190 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.699132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vmnv\" (UniqueName: \"kubernetes.io/projected/5a062f3a-343a-437a-b521-ae76e8795f27-kube-api-access-5vmnv\") pod \"downloads-5bc67dcd46-rd9tj\" (UID: \"5a062f3a-343a-437a-b521-ae76e8795f27\") " pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:36:49.772206 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.772129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjvqs" event={"ID":"7cc4025b-f7ea-4d85-99f3-c33b38439416","Type":"ContainerStarted","Data":"2e62003daa6f28bcba18d87547195966ef7b3d792519263a28aa954120074a21"}
Jul 18 13:36:49.784411 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.784381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:36:49.936587 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:49.936511    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rd9tj"]
Jul 18 13:36:49.940883 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:49.940845    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a062f3a_343a_437a_b521_ae76e8795f27.slice/crio-04d4c1478c05a9dca57ae60c334730282b8f42f14f0a82643a7200454cda7273 WatchSource:0}: Error finding container 04d4c1478c05a9dca57ae60c334730282b8f42f14f0a82643a7200454cda7273: Status 404 returned error can't find the container with id 04d4c1478c05a9dca57ae60c334730282b8f42f14f0a82643a7200454cda7273
Jul 18 13:36:50.776213 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:50.776173    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rd9tj" event={"ID":"5a062f3a-343a-437a-b521-ae76e8795f27","Type":"ContainerStarted","Data":"04d4c1478c05a9dca57ae60c334730282b8f42f14f0a82643a7200454cda7273"}
Jul 18 13:36:52.785522 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:52.785484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjvqs" event={"ID":"7cc4025b-f7ea-4d85-99f3-c33b38439416","Type":"ContainerStarted","Data":"9a576a073d49d6c2931f285ca5ea39855ecbfa8c4419f15c2f546b39e9c45b6f"}
Jul 18 13:36:54.227151 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:54.227100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:54.230254 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:54.230223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a2e8610-58a1-4df8-8f77-d207ad218433-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4584v\" (UID: \"4a2e8610-58a1-4df8-8f77-d207ad218433\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:54.473260 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:54.473212    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd" podUID="9a1d26ef-9167-4ae1-9279-fd20187cf877"
Jul 18 13:36:54.473260 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:54.473213    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fhc8x" podUID="a9bc7408-0f67-49c3-811b-44dac65c9016"
Jul 18 13:36:54.473472 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:36:54.473228    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" podUID="5711ee18-b68f-457a-a167-7b87476eed22"
Jul 18 13:36:54.510454 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:54.510378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"
Jul 18 13:36:54.792497 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:54.792425    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:36:54.792642 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:54.792428    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:36:54.792720 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:54.792427    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:36:56.357112 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:56.357090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v"]
Jul 18 13:36:56.360682 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:56.360639    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2e8610_58a1_4df8_8f77_d207ad218433.slice/crio-e228a2d6cf12ad435e707a904e825c1cf2deeaea63937eafbe406bdf0a0fcfd6 WatchSource:0}: Error finding container e228a2d6cf12ad435e707a904e825c1cf2deeaea63937eafbe406bdf0a0fcfd6: Status 404 returned error can't find the container with id e228a2d6cf12ad435e707a904e825c1cf2deeaea63937eafbe406bdf0a0fcfd6
Jul 18 13:36:56.800989 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:56.800951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v" event={"ID":"4a2e8610-58a1-4df8-8f77-d207ad218433","Type":"ContainerStarted","Data":"e228a2d6cf12ad435e707a904e825c1cf2deeaea63937eafbe406bdf0a0fcfd6"}
Jul 18 13:36:56.803194 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:56.803160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjvqs" event={"ID":"7cc4025b-f7ea-4d85-99f3-c33b38439416","Type":"ContainerStarted","Data":"252020229e18a253c2b699bc73388715a28bbb45b1a8456051f89576345c2d50"}
Jul 18 13:36:56.820802 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:56.820750    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sjvqs" podStartSLOduration=16.72317465 podStartE2EDuration="28.820729637s" totalImagesPullingTime="12.097554987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:36:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:56.819246238 +0000 UTC m=+284.488695945" watchObservedRunningTime="2026-07-18 13:36:56.820729637 +0000 UTC m=+284.490179344"
Jul 18 13:36:57.656042 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.656006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:36:57.656520 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.656054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:36:57.658954 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.658901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a1d26ef-9167-4ae1-9279-fd20187cf877-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8lpzd\" (UID: \"9a1d26ef-9167-4ae1-9279-fd20187cf877\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:36:57.659334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.659290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"image-registry-867dc7dc44-rj4x2\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") " pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:36:57.757203 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.757159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:36:57.759927 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.759885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9bc7408-0f67-49c3-811b-44dac65c9016-metrics-tls\") pod \"dns-default-fhc8x\" (UID: \"a9bc7408-0f67-49c3-811b-44dac65c9016\") " pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:36:57.796660 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.796562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xfjwq\""
Jul 18 13:36:57.796660 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.796597    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t94bf\""
Jul 18 13:36:57.796660 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.796562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-67xht\""
Jul 18 13:36:57.803631 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.803573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:36:57.803631 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.803588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:36:57.803827 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:57.803642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"
Jul 18 13:36:58.363511 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.363477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-867dc7dc44-rj4x2"]
Jul 18 13:36:58.367174 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:58.367144    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5711ee18_b68f_457a_a167_7b87476eed22.slice/crio-332f2a1f7c286758ef287dccc671bb59c75d9035f66a9f063d67e2f945d20005 WatchSource:0}: Error finding container 332f2a1f7c286758ef287dccc671bb59c75d9035f66a9f063d67e2f945d20005: Status 404 returned error can't find the container with id 332f2a1f7c286758ef287dccc671bb59c75d9035f66a9f063d67e2f945d20005
Jul 18 13:36:58.374477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.374458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd"]
Jul 18 13:36:58.377639 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.377392    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fhc8x"]
Jul 18 13:36:58.377871 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:58.377828    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1d26ef_9167_4ae1_9279_fd20187cf877.slice/crio-cba889851924b0dbcca8d8b4282b82b70865ebd68034b37e886009491fd1a731 WatchSource:0}: Error finding container cba889851924b0dbcca8d8b4282b82b70865ebd68034b37e886009491fd1a731: Status 404 returned error can't find the container with id cba889851924b0dbcca8d8b4282b82b70865ebd68034b37e886009491fd1a731
Jul 18 13:36:58.382114 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:58.382072    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9bc7408_0f67_49c3_811b_44dac65c9016.slice/crio-06388388696e78339f7983c31c232a7630a8dd9893a34e5cfb1564b709422146 WatchSource:0}: Error finding container 06388388696e78339f7983c31c232a7630a8dd9893a34e5cfb1564b709422146: Status 404 returned error can't find the container with id 06388388696e78339f7983c31c232a7630a8dd9893a34e5cfb1564b709422146
Jul 18 13:36:58.396309 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.395234    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cdc4d7c88-grmgb"]
Jul 18 13:36:58.400026 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.400003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.403145 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.402954    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 18 13:36:58.403145 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.402967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 18 13:36:58.403145 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.402989    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 18 13:36:58.403145 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.403015    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jh5bg\""
Jul 18 13:36:58.403145 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.403015    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 18 13:36:58.403145 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.403100    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 18 13:36:58.409177 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.409124    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cdc4d7c88-grmgb"]
Jul 18 13:36:58.462601 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.462580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-console-config\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.462715 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.462615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-oauth-config\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.462715 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.462658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmxlm\" (UniqueName: \"kubernetes.io/projected/2cd58c97-8e59-459b-b10b-821ffcddf600-kube-api-access-xmxlm\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.462715 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.462703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-oauth-serving-cert\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.462822 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.462734    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-serving-cert\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.462822 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.462778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-service-ca\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.563601 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.563512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-serving-cert\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.563601 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.563568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-service-ca\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.563839 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.563609    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-console-config\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.563839 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.563641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-oauth-config\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.563839 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.563711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmxlm\" (UniqueName: \"kubernetes.io/projected/2cd58c97-8e59-459b-b10b-821ffcddf600-kube-api-access-xmxlm\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.563839 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.563741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-oauth-serving-cert\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.564539 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.564516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-oauth-serving-cert\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.564630 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.564522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-service-ca\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.564630 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.564614    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-console-config\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.566687 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.566644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-oauth-config\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.567108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.567076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-serving-cert\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.572263 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.572241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmxlm\" (UniqueName: \"kubernetes.io/projected/2cd58c97-8e59-459b-b10b-821ffcddf600-kube-api-access-xmxlm\") pod \"console-5cdc4d7c88-grmgb\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") " pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.726449 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.726168    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:36:58.817492 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.817448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd" event={"ID":"9a1d26ef-9167-4ae1-9279-fd20187cf877","Type":"ContainerStarted","Data":"cba889851924b0dbcca8d8b4282b82b70865ebd68034b37e886009491fd1a731"}
Jul 18 13:36:58.821518 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.820872    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:36:58.821719 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.821700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" event={"ID":"5711ee18-b68f-457a-a167-7b87476eed22","Type":"ContainerStarted","Data":"8fbc0dbda0a308800110a8cf82b3ccf140fd2ddb47438ef497ef989ee6d3716a"}
Jul 18 13:36:58.821891 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.821873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" event={"ID":"5711ee18-b68f-457a-a167-7b87476eed22","Type":"ContainerStarted","Data":"332f2a1f7c286758ef287dccc671bb59c75d9035f66a9f063d67e2f945d20005"}
Jul 18 13:36:58.824417 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.824391    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v" event={"ID":"4a2e8610-58a1-4df8-8f77-d207ad218433","Type":"ContainerStarted","Data":"beffadb61804efe5c0b62dc6d10c7c3b0630656cd785be7001d164f42cca9233"}
Jul 18 13:36:58.825998 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.825972    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhc8x" event={"ID":"a9bc7408-0f67-49c3-811b-44dac65c9016","Type":"ContainerStarted","Data":"06388388696e78339f7983c31c232a7630a8dd9893a34e5cfb1564b709422146"}
Jul 18 13:36:58.841761 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.841709    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" podStartSLOduration=284.841690971 podStartE2EDuration="4m44.841690971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:58.840811186 +0000 UTC m=+286.510260892" watchObservedRunningTime="2026-07-18 13:36:58.841690971 +0000 UTC m=+286.511140672"
Jul 18 13:36:58.863182 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.863080    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4584v" podStartSLOduration=35.003785092 podStartE2EDuration="36.863060623s" totalImagesPullingTime="1.859275531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:58.861909066 +0000 UTC m=+286.531358771" watchObservedRunningTime="2026-07-18 13:36:58.863060623 +0000 UTC m=+286.532510315"
Jul 18 13:36:58.891518 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:58.891454    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cdc4d7c88-grmgb"]
Jul 18 13:36:58.896437 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:36:58.896397    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd58c97_8e59_459b_b10b_821ffcddf600.slice/crio-7745ca0ecb65113f0927c84967ccf095cd27dcec890eacadfcf9153360baca8d WatchSource:0}: Error finding container 7745ca0ecb65113f0927c84967ccf095cd27dcec890eacadfcf9153360baca8d: Status 404 returned error can't find the container with id 7745ca0ecb65113f0927c84967ccf095cd27dcec890eacadfcf9153360baca8d
Jul 18 13:36:59.835480 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:36:59.835381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdc4d7c88-grmgb" event={"ID":"2cd58c97-8e59-459b-b10b-821ffcddf600","Type":"ContainerStarted","Data":"7745ca0ecb65113f0927c84967ccf095cd27dcec890eacadfcf9153360baca8d"}
Jul 18 13:37:00.847043 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:00.847011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhc8x" event={"ID":"a9bc7408-0f67-49c3-811b-44dac65c9016","Type":"ContainerStarted","Data":"8b6defd75e505c814e1deb81cb56e223b0ff61add97af674255458fa16e91291"}
Jul 18 13:37:00.847438 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:00.847074    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:37:00.847438 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:00.847100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhc8x" event={"ID":"a9bc7408-0f67-49c3-811b-44dac65c9016","Type":"ContainerStarted","Data":"507d535e2acba9ac2a47810ba5a8fafd9dec8e323e14ea435304a453d64ddd40"}
Jul 18 13:37:00.847438 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:00.847121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd" event={"ID":"9a1d26ef-9167-4ae1-9279-fd20187cf877","Type":"ContainerStarted","Data":"b9f95b9b08dc3b30c543be235ca710e5e8066351fc59fa1e48c8dcaa1605437a"}
Jul 18 13:37:00.858113 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:00.858023    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fhc8x" podStartSLOduration=251.640889686 podStartE2EDuration="4m13.858009976s" totalImagesPullingTime="2.21712029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:00.857700653 +0000 UTC m=+288.527150358" watchObservedRunningTime="2026-07-18 13:37:00.858009976 +0000 UTC m=+288.527459680"
Jul 18 13:37:00.872698 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:00.872632    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8lpzd" podStartSLOduration=275.46346086 podStartE2EDuration="4m36.872613088s" totalImagesPullingTime="1.409152228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:00.870981415 +0000 UTC m=+288.540431122" watchObservedRunningTime="2026-07-18 13:37:00.872613088 +0000 UTC m=+288.542062794"
Jul 18 13:37:06.201612 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.201539    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v55jc"]
Jul 18 13:37:06.208927 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.208900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.211310 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.211225    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 13:37:06.211462 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.211438    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 13:37:06.211571 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.211484    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 13:37:06.211571 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.211544    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 13:37:06.211712 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.211695    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zg879\""
Jul 18 13:37:06.332950 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.332912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5ecce71-a288-4cfe-87ac-64dc3125281f-metrics-client-ca\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.332950 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.332949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.332971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-sys\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.332999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-root\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.333062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-tls\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.333098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-textfile\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.333114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-wtmp\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.333150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcblp\" (UniqueName: \"kubernetes.io/projected/b5ecce71-a288-4cfe-87ac-64dc3125281f-kube-api-access-pcblp\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.333530 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.333195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-accelerators-collector-config\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434526 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcblp\" (UniqueName: \"kubernetes.io/projected/b5ecce71-a288-4cfe-87ac-64dc3125281f-kube-api-access-pcblp\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434725 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-accelerators-collector-config\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434725 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5ecce71-a288-4cfe-87ac-64dc3125281f-metrics-client-ca\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434725 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434893 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-sys\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434893 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-root\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434893 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-tls\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434893 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-textfile\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.434893 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-wtmp\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.435132 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.434889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-sys\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.435132 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.435027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-wtmp\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.435132 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.435087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5ecce71-a288-4cfe-87ac-64dc3125281f-root\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.435293 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.435276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-accelerators-collector-config\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.435345 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.435290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-textfile\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.435437 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.435398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5ecce71-a288-4cfe-87ac-64dc3125281f-metrics-client-ca\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.437393 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.437363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.437542 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.437521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5ecce71-a288-4cfe-87ac-64dc3125281f-node-exporter-tls\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.442451 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.442428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcblp\" (UniqueName: \"kubernetes.io/projected/b5ecce71-a288-4cfe-87ac-64dc3125281f-kube-api-access-pcblp\") pod \"node-exporter-v55jc\" (UID: \"b5ecce71-a288-4cfe-87ac-64dc3125281f\") " pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.520125 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:06.520091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v55jc"
Jul 18 13:37:06.841982 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:37:06.841879    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t7vxw" podUID="bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8"
Jul 18 13:37:07.226527 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.226492    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 13:37:07.274401 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.274375    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 13:37:07.274548 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.274526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.277202 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277177    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 18 13:37:07.277632 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 18 13:37:07.277632 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277360    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 18 13:37:07.277632 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277411    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 18 13:37:07.277632 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277467    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 18 13:37:07.277632 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277494    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 18 13:37:07.277958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277788    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njwr7\""
Jul 18 13:37:07.277958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277912    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 18 13:37:07.278077 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277982    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 18 13:37:07.278077 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.277988    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 18 13:37:07.443916 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.443882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57997edb-1f31-4c21-921a-adbc8026498b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.443944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57997edb-1f31-4c21-921a-adbc8026498b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.443973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-config-volume\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.443996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57997edb-1f31-4c21-921a-adbc8026498b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57997edb-1f31-4c21-921a-adbc8026498b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57997edb-1f31-4c21-921a-adbc8026498b-config-out\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444334 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wxm2\" (UniqueName: \"kubernetes.io/projected/57997edb-1f31-4c21-921a-adbc8026498b-kube-api-access-9wxm2\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444562 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-web-config\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.444562 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.444417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545712 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57997edb-1f31-4c21-921a-adbc8026498b-config-out\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545712 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wxm2\" (UniqueName: \"kubernetes.io/projected/57997edb-1f31-4c21-921a-adbc8026498b-kube-api-access-9wxm2\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545712 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-web-config\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57997edb-1f31-4c21-921a-adbc8026498b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57997edb-1f31-4c21-921a-adbc8026498b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-config-volume\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57997edb-1f31-4c21-921a-adbc8026498b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.545975 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.546423 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.545996    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57997edb-1f31-4c21-921a-adbc8026498b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.546423 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.546403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57997edb-1f31-4c21-921a-adbc8026498b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.546823 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:37:07.546696    2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 18 13:37:07.546823 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:37:07.546820    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-main-tls podName:57997edb-1f31-4c21-921a-adbc8026498b nodeName:}" failed. No retries permitted until 2026-07-18 13:37:08.046797673 +0000 UTC m=+295.716247361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "57997edb-1f31-4c21-921a-adbc8026498b") : secret "alertmanager-main-tls" not found
Jul 18 13:37:07.547338 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.547286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57997edb-1f31-4c21-921a-adbc8026498b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.548053 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.548033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57997edb-1f31-4c21-921a-adbc8026498b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.550461 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.550279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.550461 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.550411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57997edb-1f31-4c21-921a-adbc8026498b-config-out\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.551509 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.551469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.551977 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.551950    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.552458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.552439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.552566 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.552546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57997edb-1f31-4c21-921a-adbc8026498b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.552613 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.552558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-web-config\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.552613 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.552586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-config-volume\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:07.554929 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:07.554908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wxm2\" (UniqueName: \"kubernetes.io/projected/57997edb-1f31-4c21-921a-adbc8026498b-kube-api-access-9wxm2\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:08.050870 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.050838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:08.053395 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.053369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57997edb-1f31-4c21-921a-adbc8026498b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57997edb-1f31-4c21-921a-adbc8026498b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:08.137621 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.137584    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"]
Jul 18 13:37:08.185877 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.185832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 13:37:08.203185 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.203157    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"]
Jul 18 13:37:08.203338 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.203264    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.206155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.205903    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-k8vcq\""
Jul 18 13:37:08.206155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.205991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 18 13:37:08.206155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.206025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 18 13:37:08.206155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.206058    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 18 13:37:08.206155 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.206079    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dpnlel07t5jjs\""
Jul 18 13:37:08.206615 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.206177    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 18 13:37:08.206615 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.206283    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 18 13:37:08.353458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57543084-fc6d-4e8d-868a-295e4b1f605a-metrics-client-ca\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-grpc-tls\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zggt\" (UniqueName: \"kubernetes.io/projected/57543084-fc6d-4e8d-868a-295e4b1f605a-kube-api-access-8zggt\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.353958 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.353776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-tls\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.454655 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.454623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.454845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.454663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.454845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.454709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.454845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.454732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57543084-fc6d-4e8d-868a-295e4b1f605a-metrics-client-ca\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.454845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.454779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-grpc-tls\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.455057 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.454962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zggt\" (UniqueName: \"kubernetes.io/projected/57543084-fc6d-4e8d-868a-295e4b1f605a-kube-api-access-8zggt\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.455057 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.455028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.455154 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.455061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-tls\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.455635 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.455569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57543084-fc6d-4e8d-868a-295e4b1f605a-metrics-client-ca\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.457890 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.457843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-grpc-tls\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.457992 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.457909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.458201 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.458183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.458356 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.458337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.458427 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.458403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.458478 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.458446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/57543084-fc6d-4e8d-868a-295e4b1f605a-secret-thanos-querier-tls\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.462894 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.462875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zggt\" (UniqueName: \"kubernetes.io/projected/57543084-fc6d-4e8d-868a-295e4b1f605a-kube-api-access-8zggt\") pod \"thanos-querier-66f8b56d9f-wktj5\" (UID: \"57543084-fc6d-4e8d-868a-295e4b1f605a\") " pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:08.513899 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:08.513869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:09.537573 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:37:09.537537    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ecce71_a288_4cfe_87ac_64dc3125281f.slice/crio-80f0853481d77daa03a6ee94b9163c2fa99bfd149944fc5f908ea31f64976d00 WatchSource:0}: Error finding container 80f0853481d77daa03a6ee94b9163c2fa99bfd149944fc5f908ea31f64976d00: Status 404 returned error can't find the container with id 80f0853481d77daa03a6ee94b9163c2fa99bfd149944fc5f908ea31f64976d00
Jul 18 13:37:09.678732 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.678705    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 13:37:09.680406 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:37:09.680345    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57997edb_1f31_4c21_921a_adbc8026498b.slice/crio-622446b709d1283e58d9696e51c03e2e22351af621fb8d00a34109bacd81fce7 WatchSource:0}: Error finding container 622446b709d1283e58d9696e51c03e2e22351af621fb8d00a34109bacd81fce7: Status 404 returned error can't find the container with id 622446b709d1283e58d9696e51c03e2e22351af621fb8d00a34109bacd81fce7
Jul 18 13:37:09.693443 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.693420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"]
Jul 18 13:37:09.696110 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:37:09.696076    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57543084_fc6d_4e8d_868a_295e4b1f605a.slice/crio-55288cfd1098b6c1d36dc767179d374936d28ab94e575c898f201993d017be35 WatchSource:0}: Error finding container 55288cfd1098b6c1d36dc767179d374936d28ab94e575c898f201993d017be35: Status 404 returned error can't find the container with id 55288cfd1098b6c1d36dc767179d374936d28ab94e575c898f201993d017be35
Jul 18 13:37:09.880204 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.880121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rd9tj" event={"ID":"5a062f3a-343a-437a-b521-ae76e8795f27","Type":"ContainerStarted","Data":"fec48501117a95fb45ee8730cc890cb12dea11cabd285de01f53d4677e7676a3"}
Jul 18 13:37:09.880362 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.880293    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:37:09.881704 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.881662    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdc4d7c88-grmgb" event={"ID":"2cd58c97-8e59-459b-b10b-821ffcddf600","Type":"ContainerStarted","Data":"cf48e0580e9af993725f84bbb5867642ef105c1eedd6fb02a8aaa691e2c20e0e"}
Jul 18 13:37:09.881816 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.881775    2579 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-rd9tj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" start-of-body=
Jul 18 13:37:09.881866 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.881822    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-rd9tj" podUID="5a062f3a-343a-437a-b521-ae76e8795f27" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused"
Jul 18 13:37:09.882817 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.882796    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"622446b709d1283e58d9696e51c03e2e22351af621fb8d00a34109bacd81fce7"}
Jul 18 13:37:09.883954 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.883931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v55jc" event={"ID":"b5ecce71-a288-4cfe-87ac-64dc3125281f","Type":"ContainerStarted","Data":"80f0853481d77daa03a6ee94b9163c2fa99bfd149944fc5f908ea31f64976d00"}
Jul 18 13:37:09.884956 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.884937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"55288cfd1098b6c1d36dc767179d374936d28ab94e575c898f201993d017be35"}
Jul 18 13:37:09.898223 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.898180    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-rd9tj" podStartSLOduration=1.238848793 podStartE2EDuration="20.898166536s" totalImagesPullingTime="19.659317743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:09.897263109 +0000 UTC m=+297.566712814" watchObservedRunningTime="2026-07-18 13:37:09.898166536 +0000 UTC m=+297.567616241"
Jul 18 13:37:09.913148 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:09.913109    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cdc4d7c88-grmgb" podStartSLOduration=1.28033905 podStartE2EDuration="11.913096238s" totalImagesPullingTime="10.632757188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:09.911901446 +0000 UTC m=+297.581351155" watchObservedRunningTime="2026-07-18 13:37:09.913096238 +0000 UTC m=+297.582545943"
Jul 18 13:37:10.852242 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:10.852209    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fhc8x"
Jul 18 13:37:10.898655 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:10.898425    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-rd9tj"
Jul 18 13:37:11.188655 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:11.188540    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cdc4d7c88-grmgb"]
Jul 18 13:37:11.897222 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:11.896721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v55jc" event={"ID":"b5ecce71-a288-4cfe-87ac-64dc3125281f","Type":"ContainerStarted","Data":"4229f4db87c4b10c2cedaff72874202ef9a622469bf0cf867feb37d766f54620"}
Jul 18 13:37:12.391279 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.391248    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 13:37:12.415386 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.415357    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 13:37:12.415553 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.415502    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.418108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.417819    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 13:37:12.418108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.417890    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 13:37:12.418108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.417909    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 13:37:12.418108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.417911    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 13:37:12.418108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.417961    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 13:37:12.418108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418074    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 13:37:12.418497 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418242    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 13:37:12.418497 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418341    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 13:37:12.418497 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 13:37:12.419278 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418776    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 13:37:12.419278 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418840    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 13:37:12.419278 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.418842    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jgxsz\""
Jul 18 13:37:12.419278 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.419029    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 13:37:12.419278 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.419055    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b1fc34a6i6r4k\""
Jul 18 13:37:12.419278 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.419103    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 18 13:37:12.419865 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.419364    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 13:37:12.424021 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.423631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 13:37:12.596625 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.596625 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.596904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a14678f-3279-4398-8783-b60c5183bc00-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.596904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4ch7\" (UniqueName: \"kubernetes.io/projected/1a14678f-3279-4398-8783-b60c5183bc00-kube-api-access-c4ch7\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.596904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.596904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.596904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.596975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a14678f-3279-4398-8783-b60c5183bc00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-config\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597193    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597216    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.597498 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.597313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698763 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698763 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a14678f-3279-4398-8783-b60c5183bc00-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4ch7\" (UniqueName: \"kubernetes.io/projected/1a14678f-3279-4398-8783-b60c5183bc00-kube-api-access-c4ch7\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.698981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.698960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a14678f-3279-4398-8783-b60c5183bc00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-config\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.699329 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.700017 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.700017 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.699446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.701338 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.701318    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a14678f-3279-4398-8783-b60c5183bc00-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.701885 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.701864    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 13:37:12.702071 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702025    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 18 13:37:12.702344 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 13:37:12.702344 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702338    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b1fc34a6i6r4k\""
Jul 18 13:37:12.702486 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702323    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 13:37:12.702486 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702359    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 13:37:12.702853 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702834    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 13:37:12.703083 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.702978    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 13:37:12.703375 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.703355    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 13:37:12.703762 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.703656    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 13:37:12.703762 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.703731    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 13:37:12.703762 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.703691    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 13:37:12.704196 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.703742    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 13:37:12.704196 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.703735    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 13:37:12.704904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.704883    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 13:37:12.708654 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.707911    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 13:37:12.711738 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.709871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4ch7\" (UniqueName: \"kubernetes.io/projected/1a14678f-3279-4398-8783-b60c5183bc00-kube-api-access-c4ch7\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.711738 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.711120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.712282 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.712260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.713925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.712637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.713925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.712734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.713925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.713141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.713925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.713199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.713925 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.713663    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a14678f-3279-4398-8783-b60c5183bc00-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.715505 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.715331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.716188 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.715728    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.716188 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.715742    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.716188 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.716045    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-config\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.716188 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.716151    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.716448 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.716424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a14678f-3279-4398-8783-b60c5183bc00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.716937 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.716902    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.717276 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.717250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.718987 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.718929    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a14678f-3279-4398-8783-b60c5183bc00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a14678f-3279-4398-8783-b60c5183bc00\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.733240 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.732883    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jgxsz\""
Jul 18 13:37:12.742348 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.742327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:12.845756 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.845732    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 13:37:13.166179 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.904604    2579 generic.go:356] "Generic (PLEG): container finished" podID="b5ecce71-a288-4cfe-87ac-64dc3125281f" containerID="4229f4db87c4b10c2cedaff72874202ef9a622469bf0cf867feb37d766f54620" exitCode=0
Jul 18 13:37:13.166179 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.904718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v55jc" event={"ID":"b5ecce71-a288-4cfe-87ac-64dc3125281f","Type":"ContainerDied","Data":"4229f4db87c4b10c2cedaff72874202ef9a622469bf0cf867feb37d766f54620"}
Jul 18 13:37:13.166179 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:12.965750    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 13:37:13.166179 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:37:12.969327    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a14678f_3279_4398_8783_b60c5183bc00.slice/crio-378227aa92fd40e843d64b5cea358bdeebd34e7ce814cda175e07f7dad4a6866 WatchSource:0}: Error finding container 378227aa92fd40e843d64b5cea358bdeebd34e7ce814cda175e07f7dad4a6866: Status 404 returned error can't find the container with id 378227aa92fd40e843d64b5cea358bdeebd34e7ce814cda175e07f7dad4a6866
Jul 18 13:37:13.912257 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.912214    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a14678f-3279-4398-8783-b60c5183bc00" containerID="7499168a7b4f583b891bba7f16abd307f3ce5ab5299869e91f1a25e089c2e45d" exitCode=0
Jul 18 13:37:13.912630 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.912592    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerDied","Data":"7499168a7b4f583b891bba7f16abd307f3ce5ab5299869e91f1a25e089c2e45d"}
Jul 18 13:37:13.912864 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.912829    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"378227aa92fd40e843d64b5cea358bdeebd34e7ce814cda175e07f7dad4a6866"}
Jul 18 13:37:13.914573 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.914543    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 13:37:13.915482 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.915158    2579 generic.go:356] "Generic (PLEG): container finished" podID="57997edb-1f31-4c21-921a-adbc8026498b" containerID="1462366eac6b185004c7db61d92d10efa11273457906b4ab76beb9ee8cbc56c7" exitCode=0
Jul 18 13:37:13.915482 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.915238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerDied","Data":"1462366eac6b185004c7db61d92d10efa11273457906b4ab76beb9ee8cbc56c7"}
Jul 18 13:37:13.920757 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.920723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v55jc" event={"ID":"b5ecce71-a288-4cfe-87ac-64dc3125281f","Type":"ContainerStarted","Data":"50dc7f2f19c6386a633eba0eabc090b51c690cc20e060df2bbacceaaba7786d5"}
Jul 18 13:37:13.920941 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.920912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v55jc" event={"ID":"b5ecce71-a288-4cfe-87ac-64dc3125281f","Type":"ContainerStarted","Data":"d646db4625b94942e19bde89ddcb2c0eaaad3799ca90925ea098c38428ea1321"}
Jul 18 13:37:13.923809 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.923773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"8a80637475cd6423cdb92ef65b9f1e469fbac91eca74831dbd6c3c1e64f79e6f"}
Jul 18 13:37:13.923908 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.923856    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"cf5b47faa82f7c24662821c136abd2d7029c2ba9cb241bef2e09e98189e052d6"}
Jul 18 13:37:13.923976 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.923911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"15653e954a83b4ad82c06f87e8df8af1d3e040ed78678eb04bd330c6db6bba4f"}
Jul 18 13:37:13.956740 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:13.956689    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v55jc" podStartSLOduration=6.252199593 podStartE2EDuration="7.956651809s" totalImagesPullingTime="1.605182669s" totalInitContainerRuntime="99.269547ms" isStatefulPod=true podCreationTimestamp="2026-07-18 13:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:13.954822774 +0000 UTC m=+301.624272483" watchObservedRunningTime="2026-07-18 13:37:13.956651809 +0000 UTC m=+301.626101514"
Jul 18 13:37:14.934428 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:14.934050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"fc59abdde98daa0c98c2644a8abbe9e324d786ed3bd49bb4f3565723fd0ef71b"}
Jul 18 13:37:14.934428 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:14.934127    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"3e23f732f1190ed931c3b290582c2adf6b925ea0dabe24caa3b8869c99e88a3b"}
Jul 18 13:37:14.934428 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:14.934173    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" event={"ID":"57543084-fc6d-4e8d-868a-295e4b1f605a","Type":"ContainerStarted","Data":"6454a353f166538d26c2a310fc2ff6e049a7e1825ae160260dc5003f95439bdc"}
Jul 18 13:37:14.956090 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:14.956040    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5" podStartSLOduration=2.616519439 podStartE2EDuration="6.956018797s" totalImagesPullingTime="4.339499358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:14.954022899 +0000 UTC m=+302.623472605" watchObservedRunningTime="2026-07-18 13:37:14.956018797 +0000 UTC m=+302.625468505"
Jul 18 13:37:15.940271 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:15.940185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:17.808186 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:17.808142    2579 patch_prober.go:28] interesting pod/image-registry-867dc7dc44-rj4x2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 18 13:37:17.808572 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:17.808209    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" podUID="5711ee18-b68f-457a-a167-7b87476eed22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 13:37:17.953009 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:17.952968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"f013878bbe9c8d143124871ad6646645d41de21fa5ea4b92a3b2239a467365a0"}
Jul 18 13:37:18.726525 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.726488    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:37:18.970981 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.970922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"6bc533f31863eb85222fdb6dc5a4b33b5b0257b67c2b9c238557f5f510ddafe1"}
Jul 18 13:37:18.971351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.971064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"dae4be78dbe284403777623b84ee100996515f3aab41cb8e9f513ba9a1ad4a3a"}
Jul 18 13:37:18.971351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.971112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"37dc18e8de63d7c5524220f5a379e900984b4c6df31d371450475525b143ff28"}
Jul 18 13:37:18.971351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.971150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"b541c33d429f65aeec79290666f75dfcb0b8fd45c13d3b656e1945a37195b8e5"}
Jul 18 13:37:18.971351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.971189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57997edb-1f31-4c21-921a-adbc8026498b","Type":"ContainerStarted","Data":"c23d24e85982d83b5760ac1c5fe8e0ab39ded116e0f1b9a899e63192560128f1"}
Jul 18 13:37:18.975114 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.975084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"3b1fcdf00f700e2a20774ebf80f12b4c9b17cba75b796ab4dc803def55a37f03"}
Jul 18 13:37:18.975244 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.975170    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"135a5befdae53278aa6e3b6fc9f47956cc94acf7c6d3eff8a6994ce9fb6bfb5a"}
Jul 18 13:37:18.975244 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.975216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"c65121b59ee332b6d0e6411e3f52d262cd4695a3dc5bcb2644844ed70b653ad5"}
Jul 18 13:37:18.975342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.975259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"08e9ff168f1c6fbba876d273a1eb9bcc405b87bbebfdee54133aa9a7620d5d98"}
Jul 18 13:37:18.975342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.975301    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"ae5e80bbcc9e065c5e4bf53c952d0bd933fda63a8c7050576b4bed108acdf151"}
Jul 18 13:37:18.975416 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.975342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a14678f-3279-4398-8783-b60c5183bc00","Type":"ContainerStarted","Data":"57413a9b5d9ab5f7d8196c565228a2f36ccdcf79b6293e5b9e007ee0b8fc1bd7"}
Jul 18 13:37:18.997118 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:18.997071    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.9571087160000005 podStartE2EDuration="11.997038459s" totalImagesPullingTime="7.014764814s" totalInitContainerRuntime="25.164929ms" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:18.995663204 +0000 UTC m=+306.665112911" watchObservedRunningTime="2026-07-18 13:37:18.997038459 +0000 UTC m=+306.666488165"
Jul 18 13:37:19.025835 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:19.024982    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.703308736 podStartE2EDuration="7.024962993s" totalImagesPullingTime="4.298539901s" totalInitContainerRuntime="23.114356ms" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:19.022656329 +0000 UTC m=+306.692106035" watchObservedRunningTime="2026-07-18 13:37:19.024962993 +0000 UTC m=+306.694412698"
Jul 18 13:37:19.840423 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:19.840394    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:37:20.833792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:20.833759    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-867dc7dc44-rj4x2"]
Jul 18 13:37:21.841864 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:21.841823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:37:21.952830 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:21.952802    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66f8b56d9f-wktj5"
Jul 18 13:37:22.742767 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:22.742732    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:37:36.924939 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:36.924896    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cdc4d7c88-grmgb" podUID="2cd58c97-8e59-459b-b10b-821ffcddf600" containerName="console" containerID="cri-o://cf48e0580e9af993725f84bbb5867642ef105c1eedd6fb02a8aaa691e2c20e0e" gracePeriod=15
Jul 18 13:37:37.045836 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.045808    2579 generic.go:356] "Generic (PLEG): container finished" podID="2cd58c97-8e59-459b-b10b-821ffcddf600" containerID="cf48e0580e9af993725f84bbb5867642ef105c1eedd6fb02a8aaa691e2c20e0e" exitCode=0
Jul 18 13:37:37.045973 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.045874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdc4d7c88-grmgb" event={"ID":"2cd58c97-8e59-459b-b10b-821ffcddf600","Type":"ContainerDied","Data":"cf48e0580e9af993725f84bbb5867642ef105c1eedd6fb02a8aaa691e2c20e0e"}
Jul 18 13:37:37.199893 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.199866    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:37:37.316264 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316236    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-oauth-config\") pod \"2cd58c97-8e59-459b-b10b-821ffcddf600\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") "
Jul 18 13:37:37.316422 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316285    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-oauth-serving-cert\") pod \"2cd58c97-8e59-459b-b10b-821ffcddf600\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") "
Jul 18 13:37:37.316422 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316308    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-console-config\") pod \"2cd58c97-8e59-459b-b10b-821ffcddf600\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") "
Jul 18 13:37:37.316422 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-service-ca\") pod \"2cd58c97-8e59-459b-b10b-821ffcddf600\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") "
Jul 18 13:37:37.316578 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316450    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmxlm\" (UniqueName: \"kubernetes.io/projected/2cd58c97-8e59-459b-b10b-821ffcddf600-kube-api-access-xmxlm\") pod \"2cd58c97-8e59-459b-b10b-821ffcddf600\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") "
Jul 18 13:37:37.316578 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316527    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-serving-cert\") pod \"2cd58c97-8e59-459b-b10b-821ffcddf600\" (UID: \"2cd58c97-8e59-459b-b10b-821ffcddf600\") "
Jul 18 13:37:37.316711 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316656    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-service-ca" (OuterVolumeSpecName: "service-ca") pod "2cd58c97-8e59-459b-b10b-821ffcddf600" (UID: "2cd58c97-8e59-459b-b10b-821ffcddf600"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:37.316795 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-console-config" (OuterVolumeSpecName: "console-config") pod "2cd58c97-8e59-459b-b10b-821ffcddf600" (UID: "2cd58c97-8e59-459b-b10b-821ffcddf600"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:37.316795 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316781    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2cd58c97-8e59-459b-b10b-821ffcddf600" (UID: "2cd58c97-8e59-459b-b10b-821ffcddf600"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:37.316896 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316871    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-oauth-serving-cert\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:37.316896 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316884    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-console-config\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:37.316896 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.316893    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd58c97-8e59-459b-b10b-821ffcddf600-service-ca\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:37.318623 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.318599    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2cd58c97-8e59-459b-b10b-821ffcddf600" (UID: "2cd58c97-8e59-459b-b10b-821ffcddf600"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:37.318747 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.318615    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2cd58c97-8e59-459b-b10b-821ffcddf600" (UID: "2cd58c97-8e59-459b-b10b-821ffcddf600"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:37.318747 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.318645    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cd58c97-8e59-459b-b10b-821ffcddf600-kube-api-access-xmxlm" (OuterVolumeSpecName: "kube-api-access-xmxlm") pod "2cd58c97-8e59-459b-b10b-821ffcddf600" (UID: "2cd58c97-8e59-459b-b10b-821ffcddf600"). InnerVolumeSpecName "kube-api-access-xmxlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:37.417441 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.417418    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmxlm\" (UniqueName: \"kubernetes.io/projected/2cd58c97-8e59-459b-b10b-821ffcddf600-kube-api-access-xmxlm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:37.417441 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.417442    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-serving-cert\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:37.417581 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:37.417452    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cd58c97-8e59-459b-b10b-821ffcddf600-console-oauth-config\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:38.051048 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:38.051019    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdc4d7c88-grmgb"
Jul 18 13:37:38.051434 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:38.051025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdc4d7c88-grmgb" event={"ID":"2cd58c97-8e59-459b-b10b-821ffcddf600","Type":"ContainerDied","Data":"7745ca0ecb65113f0927c84967ccf095cd27dcec890eacadfcf9153360baca8d"}
Jul 18 13:37:38.051434 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:38.051137    2579 scope.go:117] "RemoveContainer" containerID="cf48e0580e9af993725f84bbb5867642ef105c1eedd6fb02a8aaa691e2c20e0e"
Jul 18 13:37:38.071490 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:38.071462    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cdc4d7c88-grmgb"]
Jul 18 13:37:38.074741 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:38.074721    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cdc4d7c88-grmgb"]
Jul 18 13:37:38.846689 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:38.846646    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cd58c97-8e59-459b-b10b-821ffcddf600" path="/var/lib/kubelet/pods/2cd58c97-8e59-459b-b10b-821ffcddf600/volumes"
Jul 18 13:37:45.854299 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:45.854232    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" podUID="5711ee18-b68f-457a-a167-7b87476eed22" containerName="registry" containerID="cri-o://8fbc0dbda0a308800110a8cf82b3ccf140fd2ddb47438ef497ef989ee6d3716a" gracePeriod=30
Jul 18 13:37:46.083895 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.083864    2579 generic.go:356] "Generic (PLEG): container finished" podID="5711ee18-b68f-457a-a167-7b87476eed22" containerID="8fbc0dbda0a308800110a8cf82b3ccf140fd2ddb47438ef497ef989ee6d3716a" exitCode=0
Jul 18 13:37:46.084036 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.083916    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" event={"ID":"5711ee18-b68f-457a-a167-7b87476eed22","Type":"ContainerDied","Data":"8fbc0dbda0a308800110a8cf82b3ccf140fd2ddb47438ef497ef989ee6d3716a"}
Jul 18 13:37:46.136267 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.136219    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:37:46.290593 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-trusted-ca\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.290792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290610    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-bound-sa-token\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.290792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290637    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5711ee18-b68f-457a-a167-7b87476eed22-ca-trust-extracted\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.290792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290654    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-installation-pull-secrets\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.290792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.290792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290713    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-image-registry-private-configuration\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.290792 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-registry-certificates\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.291104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.290798    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s98b\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-kube-api-access-5s98b\") pod \"5711ee18-b68f-457a-a167-7b87476eed22\" (UID: \"5711ee18-b68f-457a-a167-7b87476eed22\") "
Jul 18 13:37:46.291104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.291057    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:46.291351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.291320    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:46.293548 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.293510    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:46.293684 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.293567    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:46.293684 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.293573    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:46.293684 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.293593    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-kube-api-access-5s98b" (OuterVolumeSpecName: "kube-api-access-5s98b") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "kube-api-access-5s98b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:46.293684 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.293636    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:46.300951 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.300925    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5711ee18-b68f-457a-a167-7b87476eed22-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5711ee18-b68f-457a-a167-7b87476eed22" (UID: "5711ee18-b68f-457a-a167-7b87476eed22"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391568    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5711ee18-b68f-457a-a167-7b87476eed22-ca-trust-extracted\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391595    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-installation-pull-secrets\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391605    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-registry-tls\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391613    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5711ee18-b68f-457a-a167-7b87476eed22-image-registry-private-configuration\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391622    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-registry-certificates\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391630    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s98b\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-kube-api-access-5s98b\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391638 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391639    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5711ee18-b68f-457a-a167-7b87476eed22-trusted-ca\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:46.391949 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:46.391648    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5711ee18-b68f-457a-a167-7b87476eed22-bound-sa-token\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:37:47.089596 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:47.089550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2" event={"ID":"5711ee18-b68f-457a-a167-7b87476eed22","Type":"ContainerDied","Data":"332f2a1f7c286758ef287dccc671bb59c75d9035f66a9f063d67e2f945d20005"}
Jul 18 13:37:47.089596 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:47.089583    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-867dc7dc44-rj4x2"
Jul 18 13:37:47.090117 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:47.089626    2579 scope.go:117] "RemoveContainer" containerID="8fbc0dbda0a308800110a8cf82b3ccf140fd2ddb47438ef497ef989ee6d3716a"
Jul 18 13:37:47.105801 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:47.105774    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-867dc7dc44-rj4x2"]
Jul 18 13:37:47.108837 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:47.108780    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-867dc7dc44-rj4x2"]
Jul 18 13:37:48.846636 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:37:48.846604    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5711ee18-b68f-457a-a167-7b87476eed22" path="/var/lib/kubelet/pods/5711ee18-b68f-457a-a167-7b87476eed22/volumes"
Jul 18 13:38:12.742534 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:12.742509    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:38:12.761162 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:12.761137    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:38:13.202845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:13.202773    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 13:38:26.627194 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:26.627146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:38:26.629424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:26.629392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8-metrics-certs\") pod \"network-metrics-daemon-t7vxw\" (UID: \"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8\") " pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:38:26.644577 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:26.644557    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l5j6b\""
Jul 18 13:38:26.652360 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:26.652347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7vxw"
Jul 18 13:38:26.773289 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:26.773224    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t7vxw"]
Jul 18 13:38:26.775482 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:38:26.775456    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc9a612f_fdf3_4dea_b467_69bf8bc5d1f8.slice/crio-4adbb8ce9c96e677a4d74e6a19625e613ea638ab16368f5555e6251a4a47d628 WatchSource:0}: Error finding container 4adbb8ce9c96e677a4d74e6a19625e613ea638ab16368f5555e6251a4a47d628: Status 404 returned error can't find the container with id 4adbb8ce9c96e677a4d74e6a19625e613ea638ab16368f5555e6251a4a47d628
Jul 18 13:38:27.240221 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:27.240182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t7vxw" event={"ID":"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8","Type":"ContainerStarted","Data":"4adbb8ce9c96e677a4d74e6a19625e613ea638ab16368f5555e6251a4a47d628"}
Jul 18 13:38:28.245300 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:28.245261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t7vxw" event={"ID":"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8","Type":"ContainerStarted","Data":"69b1cf8ede404c3cbe2906e8204509f64a341602db744ee8b233206e60a4167b"}
Jul 18 13:38:28.245816 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:28.245352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t7vxw" event={"ID":"bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8","Type":"ContainerStarted","Data":"49d62aafd95d489b5e15343dcb1f5e23f8b142c204750b6f8ddb3ef71e9b9567"}
Jul 18 13:38:28.261548 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:38:28.261492    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t7vxw" podStartSLOduration=374.357586776 podStartE2EDuration="6m15.26147774s" totalImagesPullingTime="903.890964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:38:28.259399751 +0000 UTC m=+375.928849446" watchObservedRunningTime="2026-07-18 13:38:28.26147774 +0000 UTC m=+375.930927444"
Jul 18 13:40:35.634611 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.634579    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"]
Jul 18 13:40:35.635010 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.634923    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cd58c97-8e59-459b-b10b-821ffcddf600" containerName="console"
Jul 18 13:40:35.635010 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.634935    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cd58c97-8e59-459b-b10b-821ffcddf600" containerName="console"
Jul 18 13:40:35.635010 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.634944    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5711ee18-b68f-457a-a167-7b87476eed22" containerName="registry"
Jul 18 13:40:35.635010 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.634949    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5711ee18-b68f-457a-a167-7b87476eed22" containerName="registry"
Jul 18 13:40:35.635136 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.635026    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5711ee18-b68f-457a-a167-7b87476eed22" containerName="registry"
Jul 18 13:40:35.635136 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.635035    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cd58c97-8e59-459b-b10b-821ffcddf600" containerName="console"
Jul 18 13:40:35.636905 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.636886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.639336 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.639318    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 18 13:40:35.640049 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.640033    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:40:35.640355 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.640331    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lp2dm\""
Jul 18 13:40:35.640453 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.640387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 18 13:40:35.640453 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.640402    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 18 13:40:35.640453 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.640392    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 18 13:40:35.646775 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.646753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"]
Jul 18 13:40:35.748110 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.748079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15813ff0-5384-4c75-ab1e-79dc307b838c-cert\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.748265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.748144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15813ff0-5384-4c75-ab1e-79dc307b838c-manager-config\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.748265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.748170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15813ff0-5384-4c75-ab1e-79dc307b838c-metrics-cert\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.748265 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.748200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcltz\" (UniqueName: \"kubernetes.io/projected/15813ff0-5384-4c75-ab1e-79dc307b838c-kube-api-access-jcltz\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.849022 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.848992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15813ff0-5384-4c75-ab1e-79dc307b838c-cert\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.849159 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.849046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15813ff0-5384-4c75-ab1e-79dc307b838c-manager-config\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.849159 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.849064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15813ff0-5384-4c75-ab1e-79dc307b838c-metrics-cert\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.849245 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.849168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcltz\" (UniqueName: \"kubernetes.io/projected/15813ff0-5384-4c75-ab1e-79dc307b838c-kube-api-access-jcltz\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.849801 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.849780    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15813ff0-5384-4c75-ab1e-79dc307b838c-manager-config\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.851383 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.851364    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15813ff0-5384-4c75-ab1e-79dc307b838c-cert\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.851469 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.851441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15813ff0-5384-4c75-ab1e-79dc307b838c-metrics-cert\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.860098 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.860078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcltz\" (UniqueName: \"kubernetes.io/projected/15813ff0-5384-4c75-ab1e-79dc307b838c-kube-api-access-jcltz\") pod \"lws-controller-manager-8664c48d9f-bbwc6\" (UID: \"15813ff0-5384-4c75-ab1e-79dc307b838c\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:35.945655 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:35.945593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:36.064769 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:36.064745    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"]
Jul 18 13:40:36.067132 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:40:36.067110    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15813ff0_5384_4c75_ab1e_79dc307b838c.slice/crio-280d6323a01b854a54295ccca48874f8cb1188b73d5bb68f1444886d82f0366b WatchSource:0}: Error finding container 280d6323a01b854a54295ccca48874f8cb1188b73d5bb68f1444886d82f0366b: Status 404 returned error can't find the container with id 280d6323a01b854a54295ccca48874f8cb1188b73d5bb68f1444886d82f0366b
Jul 18 13:40:36.729075 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:36.729041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6" event={"ID":"15813ff0-5384-4c75-ab1e-79dc307b838c","Type":"ContainerStarted","Data":"280d6323a01b854a54295ccca48874f8cb1188b73d5bb68f1444886d82f0366b"}
Jul 18 13:40:38.739102 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:38.739073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6" event={"ID":"15813ff0-5384-4c75-ab1e-79dc307b838c","Type":"ContainerStarted","Data":"8b8cee5019ff419d6c4f61880d2dc6456fe0aa6c04e04870223bdbc9fed6b9f2"}
Jul 18 13:40:38.739465 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:38.739154    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:38.756757 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:38.756714    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6" podStartSLOduration=1.158600413 podStartE2EDuration="3.756698976s" totalImagesPullingTime="2.598098563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:40:38.756070955 +0000 UTC m=+506.425520657" watchObservedRunningTime="2026-07-18 13:40:38.756698976 +0000 UTC m=+506.426148681"
Jul 18 13:40:40.101705 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.101657    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"]
Jul 18 13:40:40.104163 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.104146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.106869 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.106839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 13:40:40.106869 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.106837    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 13:40:40.106869 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.106846    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 13:40:40.107104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.106894    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zj28h\""
Jul 18 13:40:40.107104 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.106939    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 13:40:40.121457 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.121435    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"]
Jul 18 13:40:40.188052 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.188025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83517afa-a501-4042-9d07-9a759eb74229-tmp\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.188052 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.188056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83517afa-a501-4042-9d07-9a759eb74229-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.188269 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.188082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr4b7\" (UniqueName: \"kubernetes.io/projected/83517afa-a501-4042-9d07-9a759eb74229-kube-api-access-dr4b7\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.188269 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.188224    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83517afa-a501-4042-9d07-9a759eb74229-manifests\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.188374 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.188269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83517afa-a501-4042-9d07-9a759eb74229-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.288882 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.288853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr4b7\" (UniqueName: \"kubernetes.io/projected/83517afa-a501-4042-9d07-9a759eb74229-kube-api-access-dr4b7\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.289000 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.288910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83517afa-a501-4042-9d07-9a759eb74229-manifests\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.289000 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.288930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83517afa-a501-4042-9d07-9a759eb74229-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.289000 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.288973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83517afa-a501-4042-9d07-9a759eb74229-tmp\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.289000 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.288989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83517afa-a501-4042-9d07-9a759eb74229-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.289301 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.289280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83517afa-a501-4042-9d07-9a759eb74229-manifests\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.291772 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.291742    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83517afa-a501-4042-9d07-9a759eb74229-tmp\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.291940 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.291915    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83517afa-a501-4042-9d07-9a759eb74229-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.291988 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.291922    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83517afa-a501-4042-9d07-9a759eb74229-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.296506 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.296488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr4b7\" (UniqueName: \"kubernetes.io/projected/83517afa-a501-4042-9d07-9a759eb74229-kube-api-access-dr4b7\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-7fgxq\" (UID: \"83517afa-a501-4042-9d07-9a759eb74229\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.413793 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.413722    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:40.565115 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.565013    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"]
Jul 18 13:40:40.567993 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:40:40.567963    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83517afa_a501_4042_9d07_9a759eb74229.slice/crio-ddc0fcf61027f44437400207f3cf65031c7d2e79834f51409713346d1dc6b28f WatchSource:0}: Error finding container ddc0fcf61027f44437400207f3cf65031c7d2e79834f51409713346d1dc6b28f: Status 404 returned error can't find the container with id ddc0fcf61027f44437400207f3cf65031c7d2e79834f51409713346d1dc6b28f
Jul 18 13:40:40.748235 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:40.748202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq" event={"ID":"83517afa-a501-4042-9d07-9a759eb74229","Type":"ContainerStarted","Data":"ddc0fcf61027f44437400207f3cf65031c7d2e79834f51409713346d1dc6b28f"}
Jul 18 13:40:43.764728 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:43.764694    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq" event={"ID":"83517afa-a501-4042-9d07-9a759eb74229","Type":"ContainerStarted","Data":"cf1c01f033e673ea659b75a9862a02147cd6b63e4bbd5ac81b1bb8e7594c11ba"}
Jul 18 13:40:44.770859 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:44.770823    2579 generic.go:356] "Generic (PLEG): container finished" podID="83517afa-a501-4042-9d07-9a759eb74229" containerID="cf1c01f033e673ea659b75a9862a02147cd6b63e4bbd5ac81b1bb8e7594c11ba" exitCode=0
Jul 18 13:40:44.771252 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:44.770903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq" event={"ID":"83517afa-a501-4042-9d07-9a759eb74229","Type":"ContainerDied","Data":"cf1c01f033e673ea659b75a9862a02147cd6b63e4bbd5ac81b1bb8e7594c11ba"}
Jul 18 13:40:45.776842 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:45.776788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq" event={"ID":"83517afa-a501-4042-9d07-9a759eb74229","Type":"ContainerStarted","Data":"2359c050c556d3ac0fb601fc40a274e730b9c98878ef487b9726605c1168bba4"}
Jul 18 13:40:45.777223 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:45.776896    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:40:45.798810 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:45.798768    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq" podStartSLOduration=1.59981647 podStartE2EDuration="5.798750043s" totalImagesPullingTime="3.194087877s" totalInitContainerRuntime="1.004845696s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:40:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:40:45.797373346 +0000 UTC m=+513.466823050" watchObservedRunningTime="2026-07-18 13:40:45.798750043 +0000 UTC m=+513.468199750"
Jul 18 13:40:49.745420 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:49.745391    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-bbwc6"
Jul 18 13:40:56.783129 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:40:56.783095    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-7fgxq"
Jul 18 13:41:04.185588 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.185554    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"]
Jul 18 13:41:04.202419 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.202393    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"]
Jul 18 13:41:04.202598 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.202503    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.204948 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.204888    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 13:41:04.205870 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.205841    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 13:41:04.206011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.205840    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 13:41:04.206011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.205903    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pdp8t\""
Jul 18 13:41:04.206427 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.206410    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 13:41:04.376200 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.376165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-tls-certs\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.376376 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.376227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-tmp\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.376376 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.376270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mrzx\" (UniqueName: \"kubernetes.io/projected/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-kube-api-access-2mrzx\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.477298 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.477264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-tls-certs\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.477458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.477316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-tmp\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.477458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.477335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mrzx\" (UniqueName: \"kubernetes.io/projected/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-kube-api-access-2mrzx\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.479572 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.479542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-tmp\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.479788 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.479769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-tls-certs\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.484535 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.484515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mrzx\" (UniqueName: \"kubernetes.io/projected/14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4-kube-api-access-2mrzx\") pod \"kube-auth-proxy-7d65948f8d-px92c\" (UID: \"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4\") " pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.515386 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.515362    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"
Jul 18 13:41:04.648583 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.648559    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d65948f8d-px92c"]
Jul 18 13:41:04.651253 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:41:04.651228    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c91f45_c78d_4eb1_a5b1_1a7241fe8bd4.slice/crio-f1881f1e001647695c032765c2ae1645c48a30864afc893e9ec3b6812532576c WatchSource:0}: Error finding container f1881f1e001647695c032765c2ae1645c48a30864afc893e9ec3b6812532576c: Status 404 returned error can't find the container with id f1881f1e001647695c032765c2ae1645c48a30864afc893e9ec3b6812532576c
Jul 18 13:41:04.857571 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:04.857487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c" event={"ID":"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4","Type":"ContainerStarted","Data":"f1881f1e001647695c032765c2ae1645c48a30864afc893e9ec3b6812532576c"}
Jul 18 13:41:07.872377 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:07.872343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c" event={"ID":"14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4","Type":"ContainerStarted","Data":"7d287410ce5e899e08ef8265db0fb797502799045b657be80b94ad552b80cf3c"}
Jul 18 13:41:07.889277 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:07.889230    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d65948f8d-px92c" podStartSLOduration=1.4616369439999999 podStartE2EDuration="3.889216113s" totalImagesPullingTime="2.427579169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:41:07.887492107 +0000 UTC m=+535.556941822" watchObservedRunningTime="2026-07-18 13:41:07.889216113 +0000 UTC m=+535.558665817"
Jul 18 13:41:14.277095 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.277062    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"]
Jul 18 13:41:14.281988 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.281971    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.284908 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.284885    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8sftc\""
Jul 18 13:41:14.285058 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.285040    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 18 13:41:14.300097 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.300073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"]
Jul 18 13:41:14.352225 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.352192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6r7d\" (UniqueName: \"kubernetes.io/projected/0bc15568-b421-4094-8108-ee4c76dbb0e7-kube-api-access-d6r7d\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.352360 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.352325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bc15568-b421-4094-8108-ee4c76dbb0e7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.453193 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.453166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bc15568-b421-4094-8108-ee4c76dbb0e7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.453315 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.453198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6r7d\" (UniqueName: \"kubernetes.io/projected/0bc15568-b421-4094-8108-ee4c76dbb0e7-kube-api-access-d6r7d\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.453379 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:41:14.453321    2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 18 13:41:14.453430 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:41:14.453394    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc15568-b421-4094-8108-ee4c76dbb0e7-cert podName:0bc15568-b421-4094-8108-ee4c76dbb0e7 nodeName:}" failed. No retries permitted until 2026-07-18 13:41:14.953373034 +0000 UTC m=+542.622822719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0bc15568-b421-4094-8108-ee4c76dbb0e7-cert") pod "llmisvc-controller-manager-777c6bd7d5-p8pzv" (UID: "0bc15568-b421-4094-8108-ee4c76dbb0e7") : secret "llmisvc-webhook-server-cert" not found
Jul 18 13:41:14.474038 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.474015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6r7d\" (UniqueName: \"kubernetes.io/projected/0bc15568-b421-4094-8108-ee4c76dbb0e7-kube-api-access-d6r7d\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.957832 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.957802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bc15568-b421-4094-8108-ee4c76dbb0e7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:14.960120 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:14.960093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bc15568-b421-4094-8108-ee4c76dbb0e7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p8pzv\" (UID: \"0bc15568-b421-4094-8108-ee4c76dbb0e7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:15.192216 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:15.192189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:15.339020 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:15.338996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"]
Jul 18 13:41:15.340604 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:41:15.340573    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0bc15568_b421_4094_8108_ee4c76dbb0e7.slice/crio-8453224e6f3e761c1da3e3a2184e3e9b63ba28f2b9c56277faf8118f7babd2d8 WatchSource:0}: Error finding container 8453224e6f3e761c1da3e3a2184e3e9b63ba28f2b9c56277faf8118f7babd2d8: Status 404 returned error can't find the container with id 8453224e6f3e761c1da3e3a2184e3e9b63ba28f2b9c56277faf8118f7babd2d8
Jul 18 13:41:15.904545 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:15.904510    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv" event={"ID":"0bc15568-b421-4094-8108-ee4c76dbb0e7","Type":"ContainerStarted","Data":"8453224e6f3e761c1da3e3a2184e3e9b63ba28f2b9c56277faf8118f7babd2d8"}
Jul 18 13:41:18.920018 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:18.919985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv" event={"ID":"0bc15568-b421-4094-8108-ee4c76dbb0e7","Type":"ContainerStarted","Data":"acbf66cd72aacf1ca487d5483a9732fd9052ada83a05900d551da468f49510cb"}
Jul 18 13:41:18.920389 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:18.920120    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:41:18.954620 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:18.954441    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv" podStartSLOduration=2.195246494 podStartE2EDuration="4.954420326s" totalImagesPullingTime="2.759173832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:41:18.946481945 +0000 UTC m=+546.615931651" watchObservedRunningTime="2026-07-18 13:41:18.954420326 +0000 UTC m=+546.623870033"
Jul 18 13:41:49.926248 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:41:49.926213    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p8pzv"
Jul 18 13:43:02.423339 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.423261    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"]
Jul 18 13:43:02.426694 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.426658    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.429233 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.429206    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gfbgs\""
Jul 18 13:43:02.429345 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.429207    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 13:43:02.429345 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.429207    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 13:43:02.443111 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.443082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"]
Jul 18 13:43:02.533594 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.533563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5bcb87-f878-494c-937d-2ad8e7d25287-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.533789 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.533607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzxxl\" (UniqueName: \"kubernetes.io/projected/fb5bcb87-f878-494c-937d-2ad8e7d25287-kube-api-access-pzxxl\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.634719 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.634657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5bcb87-f878-494c-937d-2ad8e7d25287-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.634892 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.634731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzxxl\" (UniqueName: \"kubernetes.io/projected/fb5bcb87-f878-494c-937d-2ad8e7d25287-kube-api-access-pzxxl\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.635112 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.635088    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5bcb87-f878-494c-937d-2ad8e7d25287-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.643451 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.643421    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzxxl\" (UniqueName: \"kubernetes.io/projected/fb5bcb87-f878-494c-937d-2ad8e7d25287-kube-api-access-pzxxl\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.735992 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.735962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:02.869913 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.867751    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"]
Jul 18 13:43:02.874358 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:43:02.874332    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5bcb87_f878_494c_937d_2ad8e7d25287.slice/crio-05549bb1e2c2e39efd7217d73e19d749b0bae5f1c66aed436235c435b35b9663 WatchSource:0}: Error finding container 05549bb1e2c2e39efd7217d73e19d749b0bae5f1c66aed436235c435b35b9663: Status 404 returned error can't find the container with id 05549bb1e2c2e39efd7217d73e19d749b0bae5f1c66aed436235c435b35b9663
Jul 18 13:43:02.876541 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:02.876527    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 13:43:03.074388 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.074326    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"]
Jul 18 13:43:03.080142 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.080120    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"]
Jul 18 13:43:03.097152 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.097117    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"]
Jul 18 13:43:03.101398 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.101377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.116178 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.116155    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"]
Jul 18 13:43:03.139407 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.139380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13324f85-d85f-47ab-943d-4748da62a1a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qp4ng\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.139523 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.139427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k68n\" (UniqueName: \"kubernetes.io/projected/13324f85-d85f-47ab-943d-4748da62a1a8-kube-api-access-8k68n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qp4ng\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.240585 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.240552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13324f85-d85f-47ab-943d-4748da62a1a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qp4ng\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.240751 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.240612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k68n\" (UniqueName: \"kubernetes.io/projected/13324f85-d85f-47ab-943d-4748da62a1a8-kube-api-access-8k68n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qp4ng\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.240924 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.240905    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13324f85-d85f-47ab-943d-4748da62a1a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qp4ng\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.248697 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.248680    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k68n\" (UniqueName: \"kubernetes.io/projected/13324f85-d85f-47ab-943d-4748da62a1a8-kube-api-access-8k68n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qp4ng\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.412444 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.412371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:03.821875 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:03.821844    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"]
Jul 18 13:43:03.839836 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:43:03.839798    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13324f85_d85f_47ab_943d_4748da62a1a8.slice/crio-8600edc8944ea9d82e27c4f37b8a39bdc9a1f228512b2d1c5265b65f934dcc2d WatchSource:0}: Error finding container 8600edc8944ea9d82e27c4f37b8a39bdc9a1f228512b2d1c5265b65f934dcc2d: Status 404 returned error can't find the container with id 8600edc8944ea9d82e27c4f37b8a39bdc9a1f228512b2d1c5265b65f934dcc2d
Jul 18 13:43:04.377595 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:04.377556    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng" event={"ID":"13324f85-d85f-47ab-943d-4748da62a1a8","Type":"ContainerStarted","Data":"8600edc8944ea9d82e27c4f37b8a39bdc9a1f228512b2d1c5265b65f934dcc2d"}
Jul 18 13:43:07.395072 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.395040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng" event={"ID":"13324f85-d85f-47ab-943d-4748da62a1a8","Type":"ContainerStarted","Data":"1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04"}
Jul 18 13:43:07.395528 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.395105    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:07.396658 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.396628    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" containerName="manager" containerID="cri-o://2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b" gracePeriod=2
Jul 18 13:43:07.413863 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.413822    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng" podStartSLOduration=1.501085653 podStartE2EDuration="4.413807689s" totalImagesPullingTime="2.912722036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:43:07.412546504 +0000 UTC m=+655.081996209" watchObservedRunningTime="2026-07-18 13:43:07.413807689 +0000 UTC m=+655.083257391"
Jul 18 13:43:07.633571 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.633541    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:07.635563 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:43:07.635534    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" is forbidden: User \"system:node:ip-10-0-135-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-180.ec2.internal' and this object" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:07.681702 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.681623    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5bcb87-f878-494c-937d-2ad8e7d25287-extensions-socket-volume\") pod \"fb5bcb87-f878-494c-937d-2ad8e7d25287\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") "
Jul 18 13:43:07.681805 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.681727    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzxxl\" (UniqueName: \"kubernetes.io/projected/fb5bcb87-f878-494c-937d-2ad8e7d25287-kube-api-access-pzxxl\") pod \"fb5bcb87-f878-494c-937d-2ad8e7d25287\" (UID: \"fb5bcb87-f878-494c-937d-2ad8e7d25287\") "
Jul 18 13:43:07.681857 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.681836    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5bcb87-f878-494c-937d-2ad8e7d25287-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb5bcb87-f878-494c-937d-2ad8e7d25287" (UID: "fb5bcb87-f878-494c-937d-2ad8e7d25287"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:43:07.681944 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.681933    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5bcb87-f878-494c-937d-2ad8e7d25287-extensions-socket-volume\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:43:07.683724 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.683704    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5bcb87-f878-494c-937d-2ad8e7d25287-kube-api-access-pzxxl" (OuterVolumeSpecName: "kube-api-access-pzxxl") pod "fb5bcb87-f878-494c-937d-2ad8e7d25287" (UID: "fb5bcb87-f878-494c-937d-2ad8e7d25287"). InnerVolumeSpecName "kube-api-access-pzxxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:43:07.782852 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:07.782826    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzxxl\" (UniqueName: \"kubernetes.io/projected/fb5bcb87-f878-494c-937d-2ad8e7d25287-kube-api-access-pzxxl\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:43:08.408105 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:08.408068    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb5bcb87-f878-494c-937d-2ad8e7d25287" containerID="2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b" exitCode=2
Jul 18 13:43:08.408105 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:08.408106    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:08.408593 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:08.408162    2579 scope.go:117] "RemoveContainer" containerID="2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b"
Jul 18 13:43:08.410231 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:43:08.410204    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" is forbidden: User \"system:node:ip-10-0-135-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-180.ec2.internal' and this object" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:08.417428 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:08.417411    2579 scope.go:117] "RemoveContainer" containerID="2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b"
Jul 18 13:43:08.417665 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:43:08.417646    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b\": container with ID starting with 2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b not found: ID does not exist" containerID="2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b"
Jul 18 13:43:08.417727 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:08.417694    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b"} err="failed to get container status \"2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b\": rpc error: code = NotFound desc = could not find container \"2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b\": container with ID starting with 2a737a5e61245c96374c7bc1ba24a72e204afce8f2e28f9cabe232798e6df26b not found: ID does not exist"
Jul 18 13:43:08.418582 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:43:08.418562    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kdkd4\" is forbidden: User \"system:node:ip-10-0-135-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-180.ec2.internal' and this object" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kdkd4"
Jul 18 13:43:08.846458 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:08.846430    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" path="/var/lib/kubelet/pods/fb5bcb87-f878-494c-937d-2ad8e7d25287/volumes"
Jul 18 13:43:18.410200 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.410169    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:18.681951 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.681880    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"]
Jul 18 13:43:18.682240 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.682227    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" containerName="manager"
Jul 18 13:43:18.682279 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.682242    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" containerName="manager"
Jul 18 13:43:18.682323 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.682313    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5bcb87-f878-494c-937d-2ad8e7d25287" containerName="manager"
Jul 18 13:43:18.686570 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.686551    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.698219 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.698196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"]
Jul 18 13:43:18.773745 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.773708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dhn7\" (UniqueName: \"kubernetes.io/projected/485bb2f9-ae1b-40a7-871a-bb7504a68034-kube-api-access-6dhn7\") pod \"kuadrant-operator-controller-manager-86878c769f-8t254\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.773918 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.773767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/485bb2f9-ae1b-40a7-871a-bb7504a68034-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8t254\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.875208 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.875177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dhn7\" (UniqueName: \"kubernetes.io/projected/485bb2f9-ae1b-40a7-871a-bb7504a68034-kube-api-access-6dhn7\") pod \"kuadrant-operator-controller-manager-86878c769f-8t254\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.875363 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.875214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/485bb2f9-ae1b-40a7-871a-bb7504a68034-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8t254\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.875640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.875621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/485bb2f9-ae1b-40a7-871a-bb7504a68034-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8t254\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.883023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.882996    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dhn7\" (UniqueName: \"kubernetes.io/projected/485bb2f9-ae1b-40a7-871a-bb7504a68034-kube-api-access-6dhn7\") pod \"kuadrant-operator-controller-manager-86878c769f-8t254\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:18.996506 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:18.996474    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:19.130110 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:19.130076    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"]
Jul 18 13:43:19.133576 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:43:19.133542    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485bb2f9_ae1b_40a7_871a_bb7504a68034.slice/crio-211a2852687f01a4cecafca6cdff2766f7e96c0a32b55a08df55c2395517593c WatchSource:0}: Error finding container 211a2852687f01a4cecafca6cdff2766f7e96c0a32b55a08df55c2395517593c: Status 404 returned error can't find the container with id 211a2852687f01a4cecafca6cdff2766f7e96c0a32b55a08df55c2395517593c
Jul 18 13:43:19.462116 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:19.462031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254" event={"ID":"485bb2f9-ae1b-40a7-871a-bb7504a68034","Type":"ContainerStarted","Data":"787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576"}
Jul 18 13:43:19.462116 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:19.462091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254" event={"ID":"485bb2f9-ae1b-40a7-871a-bb7504a68034","Type":"ContainerStarted","Data":"211a2852687f01a4cecafca6cdff2766f7e96c0a32b55a08df55c2395517593c"}
Jul 18 13:43:19.462484 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:19.462171    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:19.485997 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:19.485742    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254" podStartSLOduration=1.4857228 podStartE2EDuration="1.4857228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:43:19.483605686 +0000 UTC m=+667.153055391" watchObservedRunningTime="2026-07-18 13:43:19.4857228 +0000 UTC m=+667.155172506"
Jul 18 13:43:30.472296 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.472261    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:43:30.518170 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.518136    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"]
Jul 18 13:43:30.518387 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.518362    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng" podUID="13324f85-d85f-47ab-943d-4748da62a1a8" containerName="manager" containerID="cri-o://1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04" gracePeriod=10
Jul 18 13:43:30.764077 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.764050    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:30.871342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.871312    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k68n\" (UniqueName: \"kubernetes.io/projected/13324f85-d85f-47ab-943d-4748da62a1a8-kube-api-access-8k68n\") pod \"13324f85-d85f-47ab-943d-4748da62a1a8\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") "
Jul 18 13:43:30.871494 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.871394    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13324f85-d85f-47ab-943d-4748da62a1a8-extensions-socket-volume\") pod \"13324f85-d85f-47ab-943d-4748da62a1a8\" (UID: \"13324f85-d85f-47ab-943d-4748da62a1a8\") "
Jul 18 13:43:30.871790 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.871765    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13324f85-d85f-47ab-943d-4748da62a1a8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "13324f85-d85f-47ab-943d-4748da62a1a8" (UID: "13324f85-d85f-47ab-943d-4748da62a1a8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:43:30.873240 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.873213    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13324f85-d85f-47ab-943d-4748da62a1a8-kube-api-access-8k68n" (OuterVolumeSpecName: "kube-api-access-8k68n") pod "13324f85-d85f-47ab-943d-4748da62a1a8" (UID: "13324f85-d85f-47ab-943d-4748da62a1a8"). InnerVolumeSpecName "kube-api-access-8k68n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:43:30.972351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.972326    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13324f85-d85f-47ab-943d-4748da62a1a8-extensions-socket-volume\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:43:30.972351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:30.972346    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k68n\" (UniqueName: \"kubernetes.io/projected/13324f85-d85f-47ab-943d-4748da62a1a8-kube-api-access-8k68n\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:43:31.518160 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.518127    2579 generic.go:356] "Generic (PLEG): container finished" podID="13324f85-d85f-47ab-943d-4748da62a1a8" containerID="1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04" exitCode=0
Jul 18 13:43:31.518611 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.518169    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"
Jul 18 13:43:31.518611 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.518215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng" event={"ID":"13324f85-d85f-47ab-943d-4748da62a1a8","Type":"ContainerDied","Data":"1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04"}
Jul 18 13:43:31.518611 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.518275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng" event={"ID":"13324f85-d85f-47ab-943d-4748da62a1a8","Type":"ContainerDied","Data":"8600edc8944ea9d82e27c4f37b8a39bdc9a1f228512b2d1c5265b65f934dcc2d"}
Jul 18 13:43:31.518611 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.518309    2579 scope.go:117] "RemoveContainer" containerID="1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04"
Jul 18 13:43:31.527608 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.527592    2579 scope.go:117] "RemoveContainer" containerID="1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04"
Jul 18 13:43:31.527880 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:43:31.527862    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04\": container with ID starting with 1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04 not found: ID does not exist" containerID="1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04"
Jul 18 13:43:31.527960 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.527889    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04"} err="failed to get container status \"1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04\": rpc error: code = NotFound desc = could not find container \"1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04\": container with ID starting with 1ed4e6ad66d1875fe83485a35e3076fb85f2376107dc6a94846bb896ebac1e04 not found: ID does not exist"
Jul 18 13:43:31.538301 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.538280    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"]
Jul 18 13:43:31.546747 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:31.546729    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qp4ng"]
Jul 18 13:43:32.846954 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:32.846924    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13324f85-d85f-47ab-943d-4748da62a1a8" path="/var/lib/kubelet/pods/13324f85-d85f-47ab-943d-4748da62a1a8/volumes"
Jul 18 13:43:49.875108 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.875079    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4b628"]
Jul 18 13:43:49.875657 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.875416    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13324f85-d85f-47ab-943d-4748da62a1a8" containerName="manager"
Jul 18 13:43:49.875657 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.875427    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="13324f85-d85f-47ab-943d-4748da62a1a8" containerName="manager"
Jul 18 13:43:49.875657 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.875479    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="13324f85-d85f-47ab-943d-4748da62a1a8" containerName="manager"
Jul 18 13:43:49.878476 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.878381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:49.880490 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.880468    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n9m9j\""
Jul 18 13:43:49.884110 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.884085    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4b628"]
Jul 18 13:43:49.935353 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:49.935316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzn89\" (UniqueName: \"kubernetes.io/projected/75806686-012c-4bfe-9f08-09885d2c671b-kube-api-access-vzn89\") pod \"authorino-58cd955675-4b628\" (UID: \"75806686-012c-4bfe-9f08-09885d2c671b\") " pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:50.024305 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.024269    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5bj4"]
Jul 18 13:43:50.027546 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.027532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:43:50.034099 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.034075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5bj4"]
Jul 18 13:43:50.036198 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.036168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzn89\" (UniqueName: \"kubernetes.io/projected/75806686-012c-4bfe-9f08-09885d2c671b-kube-api-access-vzn89\") pod \"authorino-58cd955675-4b628\" (UID: \"75806686-012c-4bfe-9f08-09885d2c671b\") " pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:50.036300 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.036204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tv4g\" (UniqueName: \"kubernetes.io/projected/48abbe86-50e1-45f6-848b-5d310faaf841-kube-api-access-6tv4g\") pod \"authorino-75d556bd77-j5bj4\" (UID: \"48abbe86-50e1-45f6-848b-5d310faaf841\") " pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:43:50.046291 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.046264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzn89\" (UniqueName: \"kubernetes.io/projected/75806686-012c-4bfe-9f08-09885d2c671b-kube-api-access-vzn89\") pod \"authorino-58cd955675-4b628\" (UID: \"75806686-012c-4bfe-9f08-09885d2c671b\") " pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:50.137523 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.137443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tv4g\" (UniqueName: \"kubernetes.io/projected/48abbe86-50e1-45f6-848b-5d310faaf841-kube-api-access-6tv4g\") pod \"authorino-75d556bd77-j5bj4\" (UID: \"48abbe86-50e1-45f6-848b-5d310faaf841\") " pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:43:50.144624 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.144595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tv4g\" (UniqueName: \"kubernetes.io/projected/48abbe86-50e1-45f6-848b-5d310faaf841-kube-api-access-6tv4g\") pod \"authorino-75d556bd77-j5bj4\" (UID: \"48abbe86-50e1-45f6-848b-5d310faaf841\") " pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:43:50.200907 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.200872    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:50.325180 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.325147    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4b628"]
Jul 18 13:43:50.328473 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:43:50.328445    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75806686_012c_4bfe_9f08_09885d2c671b.slice/crio-14984f90324ce5e0b33c8e70f0edf259aaa3fee3d637a15c8ad8467ca8f9f33e WatchSource:0}: Error finding container 14984f90324ce5e0b33c8e70f0edf259aaa3fee3d637a15c8ad8467ca8f9f33e: Status 404 returned error can't find the container with id 14984f90324ce5e0b33c8e70f0edf259aaa3fee3d637a15c8ad8467ca8f9f33e
Jul 18 13:43:50.337457 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.337437    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:43:50.460061 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.460031    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5bj4"]
Jul 18 13:43:50.463756 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:43:50.463725    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48abbe86_50e1_45f6_848b_5d310faaf841.slice/crio-7ee3c09c19b035e944f67a302b29d866d6e5499df3f21370afb7ca3ad78c1682 WatchSource:0}: Error finding container 7ee3c09c19b035e944f67a302b29d866d6e5499df3f21370afb7ca3ad78c1682: Status 404 returned error can't find the container with id 7ee3c09c19b035e944f67a302b29d866d6e5499df3f21370afb7ca3ad78c1682
Jul 18 13:43:50.602383 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.602341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4b628" event={"ID":"75806686-012c-4bfe-9f08-09885d2c671b","Type":"ContainerStarted","Data":"14984f90324ce5e0b33c8e70f0edf259aaa3fee3d637a15c8ad8467ca8f9f33e"}
Jul 18 13:43:50.603465 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:50.603431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5bj4" event={"ID":"48abbe86-50e1-45f6-848b-5d310faaf841","Type":"ContainerStarted","Data":"7ee3c09c19b035e944f67a302b29d866d6e5499df3f21370afb7ca3ad78c1682"}
Jul 18 13:43:54.631221 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:54.631177    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4b628" event={"ID":"75806686-012c-4bfe-9f08-09885d2c671b","Type":"ContainerStarted","Data":"a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e"}
Jul 18 13:43:54.632657 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:54.632630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5bj4" event={"ID":"48abbe86-50e1-45f6-848b-5d310faaf841","Type":"ContainerStarted","Data":"c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7"}
Jul 18 13:43:54.645712 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:54.645651    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4b628" podStartSLOduration=2.192230406 podStartE2EDuration="5.645636132s" totalImagesPullingTime="3.453405726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:43:54.644503526 +0000 UTC m=+702.313953232" watchObservedRunningTime="2026-07-18 13:43:54.645636132 +0000 UTC m=+702.315085837"
Jul 18 13:43:54.658800 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:54.658757    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-j5bj4" podStartSLOduration=1.351871089 podStartE2EDuration="4.658743669s" totalImagesPullingTime="3.30687258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:43:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:43:54.656709324 +0000 UTC m=+702.326159032" watchObservedRunningTime="2026-07-18 13:43:54.658743669 +0000 UTC m=+702.328193398"
Jul 18 13:43:54.683352 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:54.683326    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4b628"]
Jul 18 13:43:56.642035 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:56.641990    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4b628" podUID="75806686-012c-4bfe-9f08-09885d2c671b" containerName="authorino" containerID="cri-o://a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e" gracePeriod=30
Jul 18 13:43:56.893713 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:56.893644    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:56.992276 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:56.992250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzn89\" (UniqueName: \"kubernetes.io/projected/75806686-012c-4bfe-9f08-09885d2c671b-kube-api-access-vzn89\") pod \"75806686-012c-4bfe-9f08-09885d2c671b\" (UID: \"75806686-012c-4bfe-9f08-09885d2c671b\") "
Jul 18 13:43:56.994200 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:56.994167    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75806686-012c-4bfe-9f08-09885d2c671b-kube-api-access-vzn89" (OuterVolumeSpecName: "kube-api-access-vzn89") pod "75806686-012c-4bfe-9f08-09885d2c671b" (UID: "75806686-012c-4bfe-9f08-09885d2c671b"). InnerVolumeSpecName "kube-api-access-vzn89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:43:57.093619 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.093594    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzn89\" (UniqueName: \"kubernetes.io/projected/75806686-012c-4bfe-9f08-09885d2c671b-kube-api-access-vzn89\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:43:57.647943 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.647897    2579 generic.go:356] "Generic (PLEG): container finished" podID="75806686-012c-4bfe-9f08-09885d2c671b" containerID="a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e" exitCode=0
Jul 18 13:43:57.648320 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.647963    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4b628"
Jul 18 13:43:57.648320 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.647986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4b628" event={"ID":"75806686-012c-4bfe-9f08-09885d2c671b","Type":"ContainerDied","Data":"a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e"}
Jul 18 13:43:57.648320 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.648032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4b628" event={"ID":"75806686-012c-4bfe-9f08-09885d2c671b","Type":"ContainerDied","Data":"14984f90324ce5e0b33c8e70f0edf259aaa3fee3d637a15c8ad8467ca8f9f33e"}
Jul 18 13:43:57.648320 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.648051    2579 scope.go:117] "RemoveContainer" containerID="a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e"
Jul 18 13:43:57.657175 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.657158    2579 scope.go:117] "RemoveContainer" containerID="a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e"
Jul 18 13:43:57.657457 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:43:57.657440    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e\": container with ID starting with a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e not found: ID does not exist" containerID="a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e"
Jul 18 13:43:57.657522 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.657468    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e"} err="failed to get container status \"a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e\": rpc error: code = NotFound desc = could not find container \"a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e\": container with ID starting with a9925dff92376495f697d754a8efae767c3239191fc3e3c93ca9aaa00aefc63e not found: ID does not exist"
Jul 18 13:43:57.670703 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.670663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4b628"]
Jul 18 13:43:57.676188 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:57.676170    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4b628"]
Jul 18 13:43:58.847088 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:43:58.847059    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75806686-012c-4bfe-9f08-09885d2c671b" path="/var/lib/kubelet/pods/75806686-012c-4bfe-9f08-09885d2c671b/volumes"
Jul 18 13:44:20.122951 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.122917    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jl7c2"]
Jul 18 13:44:20.123308 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.123255    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75806686-012c-4bfe-9f08-09885d2c671b" containerName="authorino"
Jul 18 13:44:20.123308 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.123267    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="75806686-012c-4bfe-9f08-09885d2c671b" containerName="authorino"
Jul 18 13:44:20.123383 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.123341    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="75806686-012c-4bfe-9f08-09885d2c671b" containerName="authorino"
Jul 18 13:44:20.127730 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.127711    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:20.132181 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.132156    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jl7c2"]
Jul 18 13:44:20.183742 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.183706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8scg\" (UniqueName: \"kubernetes.io/projected/8236cecf-5d12-426b-b2c8-8e6561201256-kube-api-access-c8scg\") pod \"authorino-5d4b67c446-jl7c2\" (UID: \"8236cecf-5d12-426b-b2c8-8e6561201256\") " pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:20.284736 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.284698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8scg\" (UniqueName: \"kubernetes.io/projected/8236cecf-5d12-426b-b2c8-8e6561201256-kube-api-access-c8scg\") pod \"authorino-5d4b67c446-jl7c2\" (UID: \"8236cecf-5d12-426b-b2c8-8e6561201256\") " pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:20.293081 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.293048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8scg\" (UniqueName: \"kubernetes.io/projected/8236cecf-5d12-426b-b2c8-8e6561201256-kube-api-access-c8scg\") pod \"authorino-5d4b67c446-jl7c2\" (UID: \"8236cecf-5d12-426b-b2c8-8e6561201256\") " pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:20.303642 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.303616    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jl7c2"]
Jul 18 13:44:20.303891 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.303874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:20.331729 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.331701    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-4ch8s"]
Jul 18 13:44:20.335125 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.335110    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.337535 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.337515    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 18 13:44:20.342991 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.342955    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-4ch8s"]
Jul 18 13:44:20.367603 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.367574    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-4ch8s"]
Jul 18 13:44:20.367958 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:44:20.367912    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ssv9w tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-4ch8s" podUID="f622ce01-05bb-44a8-9234-31ee6e24e19b"
Jul 18 13:44:20.385768 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.385729    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f622ce01-05bb-44a8-9234-31ee6e24e19b-tls-cert\") pod \"authorino-76877994-4ch8s\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") " pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.385833 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.385820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssv9w\" (UniqueName: \"kubernetes.io/projected/f622ce01-05bb-44a8-9234-31ee6e24e19b-kube-api-access-ssv9w\") pod \"authorino-76877994-4ch8s\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") " pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.396694 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.396654    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7cf7ffbf77-w6wjj"]
Jul 18 13:44:20.401700 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.401657    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.408031 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.407977    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cf7ffbf77-w6wjj"]
Jul 18 13:44:20.436903 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.436875    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jl7c2"]
Jul 18 13:44:20.438975 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:44:20.438950    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8236cecf_5d12_426b_b2c8_8e6561201256.slice/crio-c2b02b3662b1b69f810c40acd286a118c15fe3a8ce11df615113b015b2ed3f18 WatchSource:0}: Error finding container c2b02b3662b1b69f810c40acd286a118c15fe3a8ce11df615113b015b2ed3f18: Status 404 returned error can't find the container with id c2b02b3662b1b69f810c40acd286a118c15fe3a8ce11df615113b015b2ed3f18
Jul 18 13:44:20.487232 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.487207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/885d1851-6c16-42d0-880a-2aa7e5f07c94-tls-cert\") pod \"authorino-7cf7ffbf77-w6wjj\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") " pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.487338 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.487246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f622ce01-05bb-44a8-9234-31ee6e24e19b-tls-cert\") pod \"authorino-76877994-4ch8s\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") " pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.487338 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.487316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssv9w\" (UniqueName: \"kubernetes.io/projected/f622ce01-05bb-44a8-9234-31ee6e24e19b-kube-api-access-ssv9w\") pod \"authorino-76877994-4ch8s\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") " pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.487449 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.487362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h48c\" (UniqueName: \"kubernetes.io/projected/885d1851-6c16-42d0-880a-2aa7e5f07c94-kube-api-access-7h48c\") pod \"authorino-7cf7ffbf77-w6wjj\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") " pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.489539 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.489522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f622ce01-05bb-44a8-9234-31ee6e24e19b-tls-cert\") pod \"authorino-76877994-4ch8s\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") " pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.494419 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.494398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssv9w\" (UniqueName: \"kubernetes.io/projected/f622ce01-05bb-44a8-9234-31ee6e24e19b-kube-api-access-ssv9w\") pod \"authorino-76877994-4ch8s\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") " pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.587989 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.587965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h48c\" (UniqueName: \"kubernetes.io/projected/885d1851-6c16-42d0-880a-2aa7e5f07c94-kube-api-access-7h48c\") pod \"authorino-7cf7ffbf77-w6wjj\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") " pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.588097 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.588018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/885d1851-6c16-42d0-880a-2aa7e5f07c94-tls-cert\") pod \"authorino-7cf7ffbf77-w6wjj\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") " pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.590023 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.590006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/885d1851-6c16-42d0-880a-2aa7e5f07c94-tls-cert\") pod \"authorino-7cf7ffbf77-w6wjj\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") " pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.595214 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.595197    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h48c\" (UniqueName: \"kubernetes.io/projected/885d1851-6c16-42d0-880a-2aa7e5f07c94-kube-api-access-7h48c\") pod \"authorino-7cf7ffbf77-w6wjj\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") " pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.713636 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.713565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:44:20.752620 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.752575    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.752774 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.752570    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jl7c2" event={"ID":"8236cecf-5d12-426b-b2c8-8e6561201256","Type":"ContainerStarted","Data":"c2b02b3662b1b69f810c40acd286a118c15fe3a8ce11df615113b015b2ed3f18"}
Jul 18 13:44:20.760223 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.760203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:20.836737 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.836716    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cf7ffbf77-w6wjj"]
Jul 18 13:44:20.838606 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:44:20.838582    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod885d1851_6c16_42d0_880a_2aa7e5f07c94.slice/crio-57df249a16d5c6f556a6158cf8de258f4e1174414713caad12e203b32cddfc9d WatchSource:0}: Error finding container 57df249a16d5c6f556a6158cf8de258f4e1174414713caad12e203b32cddfc9d: Status 404 returned error can't find the container with id 57df249a16d5c6f556a6158cf8de258f4e1174414713caad12e203b32cddfc9d
Jul 18 13:44:20.890448 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.890427    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssv9w\" (UniqueName: \"kubernetes.io/projected/f622ce01-05bb-44a8-9234-31ee6e24e19b-kube-api-access-ssv9w\") pod \"f622ce01-05bb-44a8-9234-31ee6e24e19b\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") "
Jul 18 13:44:20.890545 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.890499    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f622ce01-05bb-44a8-9234-31ee6e24e19b-tls-cert\") pod \"f622ce01-05bb-44a8-9234-31ee6e24e19b\" (UID: \"f622ce01-05bb-44a8-9234-31ee6e24e19b\") "
Jul 18 13:44:20.892585 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.892560    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f622ce01-05bb-44a8-9234-31ee6e24e19b-kube-api-access-ssv9w" (OuterVolumeSpecName: "kube-api-access-ssv9w") pod "f622ce01-05bb-44a8-9234-31ee6e24e19b" (UID: "f622ce01-05bb-44a8-9234-31ee6e24e19b"). InnerVolumeSpecName "kube-api-access-ssv9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:44:20.892878 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.892854    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f622ce01-05bb-44a8-9234-31ee6e24e19b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f622ce01-05bb-44a8-9234-31ee6e24e19b" (UID: "f622ce01-05bb-44a8-9234-31ee6e24e19b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:44:20.992191 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.992154    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f622ce01-05bb-44a8-9234-31ee6e24e19b-tls-cert\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:44:20.992191 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:20.992184    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssv9w\" (UniqueName: \"kubernetes.io/projected/f622ce01-05bb-44a8-9234-31ee6e24e19b-kube-api-access-ssv9w\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:44:21.758147 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.758109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jl7c2" event={"ID":"8236cecf-5d12-426b-b2c8-8e6561201256","Type":"ContainerStarted","Data":"3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253"}
Jul 18 13:44:21.758569 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.758192    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jl7c2" podUID="8236cecf-5d12-426b-b2c8-8e6561201256" containerName="authorino" containerID="cri-o://3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253" gracePeriod=30
Jul 18 13:44:21.759795 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.759770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj" event={"ID":"885d1851-6c16-42d0-880a-2aa7e5f07c94","Type":"ContainerStarted","Data":"d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa"}
Jul 18 13:44:21.759904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.759805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj" event={"ID":"885d1851-6c16-42d0-880a-2aa7e5f07c94","Type":"ContainerStarted","Data":"57df249a16d5c6f556a6158cf8de258f4e1174414713caad12e203b32cddfc9d"}
Jul 18 13:44:21.759904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.759855    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-4ch8s"
Jul 18 13:44:21.773309 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.773268    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jl7c2" podStartSLOduration=1.3368020569999999 podStartE2EDuration="1.773252721s" totalImagesPullingTime="436.450664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:44:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:44:21.771965553 +0000 UTC m=+729.441415258" watchObservedRunningTime="2026-07-18 13:44:21.773252721 +0000 UTC m=+729.442702429"
Jul 18 13:44:21.813616 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.813571    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj" podStartSLOduration=1.529577759 podStartE2EDuration="1.813556877s" totalImagesPullingTime="283.979118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:44:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:44:21.788920916 +0000 UTC m=+729.458370621" watchObservedRunningTime="2026-07-18 13:44:21.813556877 +0000 UTC m=+729.483006581"
Jul 18 13:44:21.814789 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.814767    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5bj4"]
Jul 18 13:44:21.814955 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.814935    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-j5bj4" podUID="48abbe86-50e1-45f6-848b-5d310faaf841" containerName="authorino" containerID="cri-o://c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7" gracePeriod=30
Jul 18 13:44:21.897510 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.897477    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-4ch8s"]
Jul 18 13:44:21.900593 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:21.900558    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-4ch8s"]
Jul 18 13:44:22.049129 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.049105    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:22.069694 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.069658    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:44:22.202965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.202936    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tv4g\" (UniqueName: \"kubernetes.io/projected/48abbe86-50e1-45f6-848b-5d310faaf841-kube-api-access-6tv4g\") pod \"48abbe86-50e1-45f6-848b-5d310faaf841\" (UID: \"48abbe86-50e1-45f6-848b-5d310faaf841\") "
Jul 18 13:44:22.203117 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.203023    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8scg\" (UniqueName: \"kubernetes.io/projected/8236cecf-5d12-426b-b2c8-8e6561201256-kube-api-access-c8scg\") pod \"8236cecf-5d12-426b-b2c8-8e6561201256\" (UID: \"8236cecf-5d12-426b-b2c8-8e6561201256\") "
Jul 18 13:44:22.204873 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.204849    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48abbe86-50e1-45f6-848b-5d310faaf841-kube-api-access-6tv4g" (OuterVolumeSpecName: "kube-api-access-6tv4g") pod "48abbe86-50e1-45f6-848b-5d310faaf841" (UID: "48abbe86-50e1-45f6-848b-5d310faaf841"). InnerVolumeSpecName "kube-api-access-6tv4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:44:22.204970 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.204902    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8236cecf-5d12-426b-b2c8-8e6561201256-kube-api-access-c8scg" (OuterVolumeSpecName: "kube-api-access-c8scg") pod "8236cecf-5d12-426b-b2c8-8e6561201256" (UID: "8236cecf-5d12-426b-b2c8-8e6561201256"). InnerVolumeSpecName "kube-api-access-c8scg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:44:22.304076 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.304052    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8scg\" (UniqueName: \"kubernetes.io/projected/8236cecf-5d12-426b-b2c8-8e6561201256-kube-api-access-c8scg\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:44:22.304076 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.304076    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tv4g\" (UniqueName: \"kubernetes.io/projected/48abbe86-50e1-45f6-848b-5d310faaf841-kube-api-access-6tv4g\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:44:22.770802 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.770764    2579 generic.go:356] "Generic (PLEG): container finished" podID="48abbe86-50e1-45f6-848b-5d310faaf841" containerID="c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7" exitCode=0
Jul 18 13:44:22.771194 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.770813    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-j5bj4"
Jul 18 13:44:22.771194 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.770851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5bj4" event={"ID":"48abbe86-50e1-45f6-848b-5d310faaf841","Type":"ContainerDied","Data":"c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7"}
Jul 18 13:44:22.771194 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.770892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-j5bj4" event={"ID":"48abbe86-50e1-45f6-848b-5d310faaf841","Type":"ContainerDied","Data":"7ee3c09c19b035e944f67a302b29d866d6e5499df3f21370afb7ca3ad78c1682"}
Jul 18 13:44:22.771194 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.770915    2579 scope.go:117] "RemoveContainer" containerID="c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7"
Jul 18 13:44:22.772367 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.772343    2579 generic.go:356] "Generic (PLEG): container finished" podID="8236cecf-5d12-426b-b2c8-8e6561201256" containerID="3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253" exitCode=0
Jul 18 13:44:22.772467 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.772408    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jl7c2"
Jul 18 13:44:22.772467 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.772431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jl7c2" event={"ID":"8236cecf-5d12-426b-b2c8-8e6561201256","Type":"ContainerDied","Data":"3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253"}
Jul 18 13:44:22.772532 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.772467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jl7c2" event={"ID":"8236cecf-5d12-426b-b2c8-8e6561201256","Type":"ContainerDied","Data":"c2b02b3662b1b69f810c40acd286a118c15fe3a8ce11df615113b015b2ed3f18"}
Jul 18 13:44:22.780246 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.780226    2579 scope.go:117] "RemoveContainer" containerID="c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7"
Jul 18 13:44:22.780515 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:44:22.780498    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7\": container with ID starting with c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7 not found: ID does not exist" containerID="c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7"
Jul 18 13:44:22.780568 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.780521    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7"} err="failed to get container status \"c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7\": rpc error: code = NotFound desc = could not find container \"c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7\": container with ID starting with c8cfee700dff5bb80a627cdc9b39aa199720ded7f71ad158109296aae1ecded7 not found: ID does not exist"
Jul 18 13:44:22.780568 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.780535    2579 scope.go:117] "RemoveContainer" containerID="3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253"
Jul 18 13:44:22.787619 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.787605    2579 scope.go:117] "RemoveContainer" containerID="3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253"
Jul 18 13:44:22.787894 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:44:22.787876    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253\": container with ID starting with 3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253 not found: ID does not exist" containerID="3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253"
Jul 18 13:44:22.787960 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.787900    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253"} err="failed to get container status \"3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253\": rpc error: code = NotFound desc = could not find container \"3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253\": container with ID starting with 3592430d140a0e83d18659a7551e7fa30266c8e088ff465b53995071d6ee5253 not found: ID does not exist"
Jul 18 13:44:22.790500 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.790480    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5bj4"]
Jul 18 13:44:22.794377 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.794358    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-j5bj4"]
Jul 18 13:44:22.803262 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.803243    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jl7c2"]
Jul 18 13:44:22.805365 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.805340    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jl7c2"]
Jul 18 13:44:22.846729 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.846707    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48abbe86-50e1-45f6-848b-5d310faaf841" path="/var/lib/kubelet/pods/48abbe86-50e1-45f6-848b-5d310faaf841/volumes"
Jul 18 13:44:22.847025 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.847013    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8236cecf-5d12-426b-b2c8-8e6561201256" path="/var/lib/kubelet/pods/8236cecf-5d12-426b-b2c8-8e6561201256/volumes"
Jul 18 13:44:22.847333 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:44:22.847322    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f622ce01-05bb-44a8-9234-31ee6e24e19b" path="/var/lib/kubelet/pods/f622ce01-05bb-44a8-9234-31ee6e24e19b/volumes"
Jul 18 13:45:19.793944 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.793910    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"]
Jul 18 13:45:19.794400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.794257    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8236cecf-5d12-426b-b2c8-8e6561201256" containerName="authorino"
Jul 18 13:45:19.794400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.794268    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8236cecf-5d12-426b-b2c8-8e6561201256" containerName="authorino"
Jul 18 13:45:19.794400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.794279    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48abbe86-50e1-45f6-848b-5d310faaf841" containerName="authorino"
Jul 18 13:45:19.794400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.794285    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48abbe86-50e1-45f6-848b-5d310faaf841" containerName="authorino"
Jul 18 13:45:19.794400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.794339    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8236cecf-5d12-426b-b2c8-8e6561201256" containerName="authorino"
Jul 18 13:45:19.794400 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.794350    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="48abbe86-50e1-45f6-848b-5d310faaf841" containerName="authorino"
Jul 18 13:45:19.798937 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.798908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.801087 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.801065    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 13:45:19.801217 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.801104    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 13:45:19.801938 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.801919    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6l8xf\""
Jul 18 13:45:19.801938 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.801931    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 18 13:45:19.805333 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.805314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"]
Jul 18 13:45:19.890238 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.890205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.890238 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.890241    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.890417 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.890275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.890417 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.890327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6754r\" (UniqueName: \"kubernetes.io/projected/74e04f62-8c93-4f02-852e-da3386dd5bfe-kube-api-access-6754r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.890417 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.890399    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e04f62-8c93-4f02-852e-da3386dd5bfe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.890527 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.890432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991196 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991196 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991426 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991426 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6754r\" (UniqueName: \"kubernetes.io/projected/74e04f62-8c93-4f02-852e-da3386dd5bfe-kube-api-access-6754r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991540 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e04f62-8c93-4f02-852e-da3386dd5bfe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991540 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991647 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991732 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.991892 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.991873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.993446 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.993426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e04f62-8c93-4f02-852e-da3386dd5bfe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.993919 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.993902    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e04f62-8c93-4f02-852e-da3386dd5bfe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:19.999001 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:19.998983    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6754r\" (UniqueName: \"kubernetes.io/projected/74e04f62-8c93-4f02-852e-da3386dd5bfe-kube-api-access-6754r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4\" (UID: \"74e04f62-8c93-4f02-852e-da3386dd5bfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:20.110976 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:20.110891    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:20.251351 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:20.251328    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"]
Jul 18 13:45:20.253224 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:45:20.253197    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e04f62_8c93_4f02_852e_da3386dd5bfe.slice/crio-2e0b2457ef738e45d361da68cc758694758050308b2b3341d1160c981dec78a9 WatchSource:0}: Error finding container 2e0b2457ef738e45d361da68cc758694758050308b2b3341d1160c981dec78a9: Status 404 returned error can't find the container with id 2e0b2457ef738e45d361da68cc758694758050308b2b3341d1160c981dec78a9
Jul 18 13:45:21.023382 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:21.023344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4" event={"ID":"74e04f62-8c93-4f02-852e-da3386dd5bfe","Type":"ContainerStarted","Data":"2e0b2457ef738e45d361da68cc758694758050308b2b3341d1160c981dec78a9"}
Jul 18 13:45:22.029825 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:22.029792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4" event={"ID":"74e04f62-8c93-4f02-852e-da3386dd5bfe","Type":"ContainerStarted","Data":"e0aaab74282e888e213de7b842786d1905272273f5965a89004e8a154b9382c2"}
Jul 18 13:45:22.047011 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:22.046964    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4" podStartSLOduration=1.900810171 podStartE2EDuration="3.046948185s" totalImagesPullingTime="1.146138014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:45:22.04470426 +0000 UTC m=+789.714153964" watchObservedRunningTime="2026-07-18 13:45:22.046948185 +0000 UTC m=+789.716397889"
Jul 18 13:45:30.111399 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:30.111367    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:30.111848 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:30.111418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:30.124353 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:30.124329    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:31.078376 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:31.078348    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4"
Jul 18 13:45:49.580554 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.580465    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"]
Jul 18 13:45:49.587777 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.587751    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.589906 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.589872    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 18 13:45:49.593512 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.593485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"]
Jul 18 13:45:49.642649 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.642617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.642827 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.642654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.642827 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.642766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.642827 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.642802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.642965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.642838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72blm\" (UniqueName: \"kubernetes.io/projected/44afb505-930d-4fe9-bfab-72281bc7191c-kube-api-access-72blm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.642965 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.642889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44afb505-930d-4fe9-bfab-72281bc7191c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744088 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744088 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744154    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72blm\" (UniqueName: \"kubernetes.io/projected/44afb505-930d-4fe9-bfab-72281bc7191c-kube-api-access-72blm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44afb505-930d-4fe9-bfab-72281bc7191c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744342 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744550 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744491    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744550 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744517    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.744730 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.744706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.746532 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.746506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44afb505-930d-4fe9-bfab-72281bc7191c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.746914 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.746892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44afb505-930d-4fe9-bfab-72281bc7191c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.751316 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.751293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72blm\" (UniqueName: \"kubernetes.io/projected/44afb505-930d-4fe9-bfab-72281bc7191c-kube-api-access-72blm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt\" (UID: \"44afb505-930d-4fe9-bfab-72281bc7191c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:49.900389 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:49.900315    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:50.030583 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:50.030560    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"]
Jul 18 13:45:50.033642 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:45:50.033609    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44afb505_930d_4fe9_bfab_72281bc7191c.slice/crio-4c4e92d9996bc6c108e73d20348366a231df8641ffd51291012f27e47260435f WatchSource:0}: Error finding container 4c4e92d9996bc6c108e73d20348366a231df8641ffd51291012f27e47260435f: Status 404 returned error can't find the container with id 4c4e92d9996bc6c108e73d20348366a231df8641ffd51291012f27e47260435f
Jul 18 13:45:50.169612 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:50.169524    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt" event={"ID":"44afb505-930d-4fe9-bfab-72281bc7191c","Type":"ContainerStarted","Data":"4c4e92d9996bc6c108e73d20348366a231df8641ffd51291012f27e47260435f"}
Jul 18 13:45:51.175472 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:51.175428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt" event={"ID":"44afb505-930d-4fe9-bfab-72281bc7191c","Type":"ContainerStarted","Data":"87dbc61078998b6092025fe33eadaa7c877fe6f6851c94bc15e00a0ac936e94f"}
Jul 18 13:45:51.192411 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:51.192360    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt" podStartSLOduration=2.01851654 podStartE2EDuration="2.19234479s" totalImagesPullingTime="173.82825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:45:51.19089094 +0000 UTC m=+818.860340645" watchObservedRunningTime="2026-07-18 13:45:51.19234479 +0000 UTC m=+818.861794503"
Jul 18 13:45:59.901434 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:59.901398    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:59.901831 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:59.901462    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:45:59.914103 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:45:59.914077    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:46:00.225973 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:00.225952    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt"
Jul 18 13:46:21.988162 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:21.988129    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"]
Jul 18 13:46:21.991969 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:21.991943    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:21.994502 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:21.994478    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 18 13:46:22.003799 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.003777    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"]
Jul 18 13:46:22.025044 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.025003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.025156 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.025046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.025156 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.025077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ln4\" (UniqueName: \"kubernetes.io/projected/6cd6c315-9cc8-4757-b211-af3f7fab8f81-kube-api-access-95ln4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.025156 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.025133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.025254 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.025168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.025254 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.025204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cd6c315-9cc8-4757-b211-af3f7fab8f81-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.125984 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.125943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.125984 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.125989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126177 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ln4\" (UniqueName: \"kubernetes.io/projected/6cd6c315-9cc8-4757-b211-af3f7fab8f81-kube-api-access-95ln4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126177 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126177 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126177 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cd6c315-9cc8-4757-b211-af3f7fab8f81-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126397 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126454 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.126454 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.126436    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.128301 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.128282    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cd6c315-9cc8-4757-b211-af3f7fab8f81-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.128640 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.128621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cd6c315-9cc8-4757-b211-af3f7fab8f81-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.133862 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.133842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ln4\" (UniqueName: \"kubernetes.io/projected/6cd6c315-9cc8-4757-b211-af3f7fab8f81-kube-api-access-95ln4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs\" (UID: \"6cd6c315-9cc8-4757-b211-af3f7fab8f81\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.301306 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.301233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:22.428386 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:22.428353    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"]
Jul 18 13:46:22.431810 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:46:22.431782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cd6c315_9cc8_4757_b211_af3f7fab8f81.slice/crio-e250ff7cfd94866523270a2b73f4c614e3f09214a74fd9f5f47c25b6a85a1876 WatchSource:0}: Error finding container e250ff7cfd94866523270a2b73f4c614e3f09214a74fd9f5f47c25b6a85a1876: Status 404 returned error can't find the container with id e250ff7cfd94866523270a2b73f4c614e3f09214a74fd9f5f47c25b6a85a1876
Jul 18 13:46:23.326544 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:23.326504    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs" event={"ID":"6cd6c315-9cc8-4757-b211-af3f7fab8f81","Type":"ContainerStarted","Data":"00869a4744016b7cb379d922e51889c80271e334c950b7a4261c2506d0d3d74a"}
Jul 18 13:46:23.326907 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:23.326552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs" event={"ID":"6cd6c315-9cc8-4757-b211-af3f7fab8f81","Type":"ContainerStarted","Data":"e250ff7cfd94866523270a2b73f4c614e3f09214a74fd9f5f47c25b6a85a1876"}
Jul 18 13:46:23.343571 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:23.343521    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs" podStartSLOduration=2.186645755 podStartE2EDuration="2.343505145s" totalImagesPullingTime="156.85939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:46:23.342270445 +0000 UTC m=+851.011720152" watchObservedRunningTime="2026-07-18 13:46:23.343505145 +0000 UTC m=+851.012954849"
Jul 18 13:46:32.301456 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:32.301421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:32.301890 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:32.301627    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:32.313898 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:32.313876    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:46:32.383159 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:46:32.383125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs"
Jul 18 13:47:04.235435 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.235400    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-559fc857d4-bjpr8"]
Jul 18 13:47:04.238910 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.238891    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.245434 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.245406    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-559fc857d4-bjpr8"]
Jul 18 13:47:04.397403 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.397362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl5nw\" (UniqueName: \"kubernetes.io/projected/1a33ac65-e196-4253-a2ac-da78bca23cd5-kube-api-access-rl5nw\") pod \"authorino-559fc857d4-bjpr8\" (UID: \"1a33ac65-e196-4253-a2ac-da78bca23cd5\") " pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.397593 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.397512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a33ac65-e196-4253-a2ac-da78bca23cd5-tls-cert\") pod \"authorino-559fc857d4-bjpr8\" (UID: \"1a33ac65-e196-4253-a2ac-da78bca23cd5\") " pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.498830 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.498734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a33ac65-e196-4253-a2ac-da78bca23cd5-tls-cert\") pod \"authorino-559fc857d4-bjpr8\" (UID: \"1a33ac65-e196-4253-a2ac-da78bca23cd5\") " pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.498830 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.498796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl5nw\" (UniqueName: \"kubernetes.io/projected/1a33ac65-e196-4253-a2ac-da78bca23cd5-kube-api-access-rl5nw\") pod \"authorino-559fc857d4-bjpr8\" (UID: \"1a33ac65-e196-4253-a2ac-da78bca23cd5\") " pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.501260 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.501235    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a33ac65-e196-4253-a2ac-da78bca23cd5-tls-cert\") pod \"authorino-559fc857d4-bjpr8\" (UID: \"1a33ac65-e196-4253-a2ac-da78bca23cd5\") " pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.505914 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.505893    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl5nw\" (UniqueName: \"kubernetes.io/projected/1a33ac65-e196-4253-a2ac-da78bca23cd5-kube-api-access-rl5nw\") pod \"authorino-559fc857d4-bjpr8\" (UID: \"1a33ac65-e196-4253-a2ac-da78bca23cd5\") " pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.569904 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.569870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-559fc857d4-bjpr8"
Jul 18 13:47:04.703346 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:04.703321    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-559fc857d4-bjpr8"]
Jul 18 13:47:04.706096 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:47:04.706068    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a33ac65_e196_4253_a2ac_da78bca23cd5.slice/crio-bd8caebd9f57efec96e4015b93d2b7ca7c4ffb42fe1b97f9279e961dd06ead8a WatchSource:0}: Error finding container bd8caebd9f57efec96e4015b93d2b7ca7c4ffb42fe1b97f9279e961dd06ead8a: Status 404 returned error can't find the container with id bd8caebd9f57efec96e4015b93d2b7ca7c4ffb42fe1b97f9279e961dd06ead8a
Jul 18 13:47:05.533574 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.533485    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-559fc857d4-bjpr8" event={"ID":"1a33ac65-e196-4253-a2ac-da78bca23cd5","Type":"ContainerStarted","Data":"e9c0c33e285464d30126928da3787a4e1ff490fcbad11c593c65ac48a1740f99"}
Jul 18 13:47:05.533574 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.533547    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-559fc857d4-bjpr8" event={"ID":"1a33ac65-e196-4253-a2ac-da78bca23cd5","Type":"ContainerStarted","Data":"bd8caebd9f57efec96e4015b93d2b7ca7c4ffb42fe1b97f9279e961dd06ead8a"}
Jul 18 13:47:05.574983 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.574914    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-559fc857d4-bjpr8" podStartSLOduration=1.103217331 podStartE2EDuration="1.574891168s" totalImagesPullingTime="471.673837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:47:05.551719234 +0000 UTC m=+893.221168939" watchObservedRunningTime="2026-07-18 13:47:05.574891168 +0000 UTC m=+893.244340874"
Jul 18 13:47:05.576258 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.576231    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7cf7ffbf77-w6wjj"]
Jul 18 13:47:05.576660 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.576621    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj" podUID="885d1851-6c16-42d0-880a-2aa7e5f07c94" containerName="authorino" containerID="cri-o://d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa" gracePeriod=30
Jul 18 13:47:05.834193 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.834162    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:47:05.913601 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.913560    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h48c\" (UniqueName: \"kubernetes.io/projected/885d1851-6c16-42d0-880a-2aa7e5f07c94-kube-api-access-7h48c\") pod \"885d1851-6c16-42d0-880a-2aa7e5f07c94\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") "
Jul 18 13:47:05.913786 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.913617    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/885d1851-6c16-42d0-880a-2aa7e5f07c94-tls-cert\") pod \"885d1851-6c16-42d0-880a-2aa7e5f07c94\" (UID: \"885d1851-6c16-42d0-880a-2aa7e5f07c94\") "
Jul 18 13:47:05.915765 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.915723    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/885d1851-6c16-42d0-880a-2aa7e5f07c94-kube-api-access-7h48c" (OuterVolumeSpecName: "kube-api-access-7h48c") pod "885d1851-6c16-42d0-880a-2aa7e5f07c94" (UID: "885d1851-6c16-42d0-880a-2aa7e5f07c94"). InnerVolumeSpecName "kube-api-access-7h48c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:47:05.924536 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:05.924509    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/885d1851-6c16-42d0-880a-2aa7e5f07c94-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "885d1851-6c16-42d0-880a-2aa7e5f07c94" (UID: "885d1851-6c16-42d0-880a-2aa7e5f07c94"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:47:06.014947 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.014905    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h48c\" (UniqueName: \"kubernetes.io/projected/885d1851-6c16-42d0-880a-2aa7e5f07c94-kube-api-access-7h48c\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:47:06.014947 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.014933    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/885d1851-6c16-42d0-880a-2aa7e5f07c94-tls-cert\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:47:06.540219 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.540133    2579 generic.go:356] "Generic (PLEG): container finished" podID="885d1851-6c16-42d0-880a-2aa7e5f07c94" containerID="d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa" exitCode=0
Jul 18 13:47:06.540219 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.540180    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj"
Jul 18 13:47:06.540219 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.540207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj" event={"ID":"885d1851-6c16-42d0-880a-2aa7e5f07c94","Type":"ContainerDied","Data":"d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa"}
Jul 18 13:47:06.540785 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.540253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cf7ffbf77-w6wjj" event={"ID":"885d1851-6c16-42d0-880a-2aa7e5f07c94","Type":"ContainerDied","Data":"57df249a16d5c6f556a6158cf8de258f4e1174414713caad12e203b32cddfc9d"}
Jul 18 13:47:06.540785 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.540274    2579 scope.go:117] "RemoveContainer" containerID="d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa"
Jul 18 13:47:06.550354 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.550334    2579 scope.go:117] "RemoveContainer" containerID="d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa"
Jul 18 13:47:06.550605 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:47:06.550584    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa\": container with ID starting with d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa not found: ID does not exist" containerID="d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa"
Jul 18 13:47:06.550691 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.550618    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa"} err="failed to get container status \"d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa\": rpc error: code = NotFound desc = could not find container \"d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa\": container with ID starting with d20b745a8da7506c898789e7daaf0cb95bc05c309484a0d79e010fe3c265f5aa not found: ID does not exist"
Jul 18 13:47:06.561926 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.561896    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7cf7ffbf77-w6wjj"]
Jul 18 13:47:06.565942 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.565919    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7cf7ffbf77-w6wjj"]
Jul 18 13:47:06.849218 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:47:06.849142    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="885d1851-6c16-42d0-880a-2aa7e5f07c94" path="/var/lib/kubelet/pods/885d1851-6c16-42d0-880a-2aa7e5f07c94/volumes"
Jul 18 13:54:14.989256 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:14.989215    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"]
Jul 18 13:54:14.991930 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:14.989513    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254" podUID="485bb2f9-ae1b-40a7-871a-bb7504a68034" containerName="manager" containerID="cri-o://787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576" gracePeriod=10
Jul 18 13:54:18.339998 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.339976    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:54:18.400608 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.400533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dhn7\" (UniqueName: \"kubernetes.io/projected/485bb2f9-ae1b-40a7-871a-bb7504a68034-kube-api-access-6dhn7\") pod \"485bb2f9-ae1b-40a7-871a-bb7504a68034\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") "
Jul 18 13:54:18.400608 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.400577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/485bb2f9-ae1b-40a7-871a-bb7504a68034-extensions-socket-volume\") pod \"485bb2f9-ae1b-40a7-871a-bb7504a68034\" (UID: \"485bb2f9-ae1b-40a7-871a-bb7504a68034\") "
Jul 18 13:54:18.401070 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.401037    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485bb2f9-ae1b-40a7-871a-bb7504a68034-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "485bb2f9-ae1b-40a7-871a-bb7504a68034" (UID: "485bb2f9-ae1b-40a7-871a-bb7504a68034"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:54:18.402540 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.402519    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/485bb2f9-ae1b-40a7-871a-bb7504a68034-kube-api-access-6dhn7" (OuterVolumeSpecName: "kube-api-access-6dhn7") pod "485bb2f9-ae1b-40a7-871a-bb7504a68034" (UID: "485bb2f9-ae1b-40a7-871a-bb7504a68034"). InnerVolumeSpecName "kube-api-access-6dhn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:54:18.501974 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.501937    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dhn7\" (UniqueName: \"kubernetes.io/projected/485bb2f9-ae1b-40a7-871a-bb7504a68034-kube-api-access-6dhn7\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:54:18.501974 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.501967    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/485bb2f9-ae1b-40a7-871a-bb7504a68034-extensions-socket-volume\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:54:18.548881 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.548842    2579 generic.go:356] "Generic (PLEG): container finished" podID="485bb2f9-ae1b-40a7-871a-bb7504a68034" containerID="787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576" exitCode=0
Jul 18 13:54:18.548881 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.548880    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"
Jul 18 13:54:18.549067 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.548921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254" event={"ID":"485bb2f9-ae1b-40a7-871a-bb7504a68034","Type":"ContainerDied","Data":"787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576"}
Jul 18 13:54:18.549067 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.548980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254" event={"ID":"485bb2f9-ae1b-40a7-871a-bb7504a68034","Type":"ContainerDied","Data":"211a2852687f01a4cecafca6cdff2766f7e96c0a32b55a08df55c2395517593c"}
Jul 18 13:54:18.549067 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.549003    2579 scope.go:117] "RemoveContainer" containerID="787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576"
Jul 18 13:54:18.558921 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.558902    2579 scope.go:117] "RemoveContainer" containerID="787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576"
Jul 18 13:54:18.559163 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:54:18.559146    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576\": container with ID starting with 787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576 not found: ID does not exist" containerID="787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576"
Jul 18 13:54:18.559233 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.559169    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576"} err="failed to get container status \"787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576\": rpc error: code = NotFound desc = could not find container \"787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576\": container with ID starting with 787198693d60fed44d47024cf17204c40a21e36d3d362708923cd83381206576 not found: ID does not exist"
Jul 18 13:54:18.570259 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.570238    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"]
Jul 18 13:54:18.575361 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.575340    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8t254"]
Jul 18 13:54:18.847453 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:54:18.847419    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="485bb2f9-ae1b-40a7-871a-bb7504a68034" path="/var/lib/kubelet/pods/485bb2f9-ae1b-40a7-871a-bb7504a68034/volumes"
Jul 18 13:55:16.083322 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083285    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"]
Jul 18 13:55:16.083718 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083629    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="485bb2f9-ae1b-40a7-871a-bb7504a68034" containerName="manager"
Jul 18 13:55:16.083718 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083640    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="485bb2f9-ae1b-40a7-871a-bb7504a68034" containerName="manager"
Jul 18 13:55:16.083718 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083661    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="885d1851-6c16-42d0-880a-2aa7e5f07c94" containerName="authorino"
Jul 18 13:55:16.083718 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083679    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="885d1851-6c16-42d0-880a-2aa7e5f07c94" containerName="authorino"
Jul 18 13:55:16.083849 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083735    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="485bb2f9-ae1b-40a7-871a-bb7504a68034" containerName="manager"
Jul 18 13:55:16.083849 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.083744    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="885d1851-6c16-42d0-880a-2aa7e5f07c94" containerName="authorino"
Jul 18 13:55:16.087760 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.087741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.090051 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.090024    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gfbgs\""
Jul 18 13:55:16.099519 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.099499    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"]
Jul 18 13:55:16.202864 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.202825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggk76\" (UniqueName: \"kubernetes.io/projected/4bbf24cc-282e-4617-874e-acfc7403f7e6-kube-api-access-ggk76\") pod \"kuadrant-operator-controller-manager-86878c769f-68zm6\" (UID: \"4bbf24cc-282e-4617-874e-acfc7403f7e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.203031 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.202905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bbf24cc-282e-4617-874e-acfc7403f7e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-68zm6\" (UID: \"4bbf24cc-282e-4617-874e-acfc7403f7e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.304105 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.304063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggk76\" (UniqueName: \"kubernetes.io/projected/4bbf24cc-282e-4617-874e-acfc7403f7e6-kube-api-access-ggk76\") pod \"kuadrant-operator-controller-manager-86878c769f-68zm6\" (UID: \"4bbf24cc-282e-4617-874e-acfc7403f7e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.304254 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.304130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bbf24cc-282e-4617-874e-acfc7403f7e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-68zm6\" (UID: \"4bbf24cc-282e-4617-874e-acfc7403f7e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.304477 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.304458    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bbf24cc-282e-4617-874e-acfc7403f7e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-68zm6\" (UID: \"4bbf24cc-282e-4617-874e-acfc7403f7e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.317450 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.317418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggk76\" (UniqueName: \"kubernetes.io/projected/4bbf24cc-282e-4617-874e-acfc7403f7e6-kube-api-access-ggk76\") pod \"kuadrant-operator-controller-manager-86878c769f-68zm6\" (UID: \"4bbf24cc-282e-4617-874e-acfc7403f7e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.397214 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.397146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.531740 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.531706    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"]
Jul 18 13:55:16.535128 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:55:16.535099    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbf24cc_282e_4617_874e_acfc7403f7e6.slice/crio-1fb96c3a7a5f0edeeb07b8ea7e0a860d8c72f63af5f502aac4f219732597f9d8 WatchSource:0}: Error finding container 1fb96c3a7a5f0edeeb07b8ea7e0a860d8c72f63af5f502aac4f219732597f9d8: Status 404 returned error can't find the container with id 1fb96c3a7a5f0edeeb07b8ea7e0a860d8c72f63af5f502aac4f219732597f9d8
Jul 18 13:55:16.814959 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.814918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6" event={"ID":"4bbf24cc-282e-4617-874e-acfc7403f7e6","Type":"ContainerStarted","Data":"967aa1bc0b3b6d001fed2a5b35e2876383b9ea0f4a66e2fae7476fbc77c795ac"}
Jul 18 13:55:16.815144 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.814977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6" event={"ID":"4bbf24cc-282e-4617-874e-acfc7403f7e6","Type":"ContainerStarted","Data":"1fb96c3a7a5f0edeeb07b8ea7e0a860d8c72f63af5f502aac4f219732597f9d8"}
Jul 18 13:55:16.815144 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.815087    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:55:16.833874 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:16.833823    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6" podStartSLOduration=0.83380837 podStartE2EDuration="833.80837ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:55:16.832355943 +0000 UTC m=+1384.501805647" watchObservedRunningTime="2026-07-18 13:55:16.83380837 +0000 UTC m=+1384.503258129"
Jul 18 13:55:27.821592 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:55:27.821560    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-68zm6"
Jul 18 13:58:03.692203 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.692128    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"]
Jul 18 13:58:03.695796 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.695777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.698263 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.698239    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-a9d318aa-kserve-self-signed-certs\""
Jul 18 13:58:03.710838 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.710816    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"]
Jul 18 13:58:03.852541 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.852508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-dshm\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.852728 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.852571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tls-certs\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.852728 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.852591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mvhc\" (UniqueName: \"kubernetes.io/projected/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-kube-api-access-6mvhc\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.852728 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.852610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-model-cache\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.852728 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.852631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-home\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.852728 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.852707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tmp-dir\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.953917 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.953837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-home\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954053 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.953917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tmp-dir\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954053 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.953980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-dshm\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954144 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.954063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tls-certs\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954144 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.954095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mvhc\" (UniqueName: \"kubernetes.io/projected/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-kube-api-access-6mvhc\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954144 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.954126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-model-cache\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954285 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.954253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-home\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954340 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.954289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tmp-dir\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.954478 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.954456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-model-cache\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.956086 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.956059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-dshm\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.956564 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.956547    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tls-certs\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:03.961425 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:03.961397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mvhc\" (UniqueName: \"kubernetes.io/projected/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-kube-api-access-6mvhc\") pod \"e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") " pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:04.004405 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:04.004376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:04.133774 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:04.133732    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"]
Jul 18 13:58:04.136574 ip-10-0-135-180 kubenswrapper[2579]: W0718 13:58:04.136541    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fa91f49_c3e5_497a_adae_535b1fa9b7fb.slice/crio-6b52ecc037f65dfe422adfc885651289ba4ebb77139961706ddb24ed3ee6c5f9 WatchSource:0}: Error finding container 6b52ecc037f65dfe422adfc885651289ba4ebb77139961706ddb24ed3ee6c5f9: Status 404 returned error can't find the container with id 6b52ecc037f65dfe422adfc885651289ba4ebb77139961706ddb24ed3ee6c5f9
Jul 18 13:58:04.595708 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:04.595653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf" event={"ID":"2fa91f49-c3e5-497a-adae-535b1fa9b7fb","Type":"ContainerStarted","Data":"f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f"}
Jul 18 13:58:04.595708 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:04.595714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf" event={"ID":"2fa91f49-c3e5-497a-adae-535b1fa9b7fb","Type":"ContainerStarted","Data":"6b52ecc037f65dfe422adfc885651289ba4ebb77139961706ddb24ed3ee6c5f9"}
Jul 18 13:58:04.612304 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:04.612252    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf" podStartSLOduration=1.612235573 podStartE2EDuration="1.612235573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:58:04.610572395 +0000 UTC m=+1552.280022100" watchObservedRunningTime="2026-07-18 13:58:04.612235573 +0000 UTC m=+1552.281685278"
Jul 18 13:58:14.004682 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:14.004642    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:14.005121 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:14.004718    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:14.017576 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:14.017552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:14.655401 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:14.655371    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:38.915357 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:38.915323    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"]
Jul 18 13:58:53.937295 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:53.937247    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf" podUID="2fa91f49-c3e5-497a-adae-535b1fa9b7fb" containerName="main" containerID="cri-o://f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f" gracePeriod=45
Jul 18 13:58:54.183583 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.183560    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:54.297575 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297542    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-model-cache\") pod \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") "
Jul 18 13:58:54.297778 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297598    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tmp-dir\") pod \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") "
Jul 18 13:58:54.297778 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-dshm\") pod \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") "
Jul 18 13:58:54.297778 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297694    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tls-certs\") pod \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") "
Jul 18 13:58:54.297778 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-home\") pod \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") "
Jul 18 13:58:54.297778 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297776    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mvhc\" (UniqueName: \"kubernetes.io/projected/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-kube-api-access-6mvhc\") pod \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\" (UID: \"2fa91f49-c3e5-497a-adae-535b1fa9b7fb\") "
Jul 18 13:58:54.298049 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297863    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-model-cache" (OuterVolumeSpecName: "model-cache") pod "2fa91f49-c3e5-497a-adae-535b1fa9b7fb" (UID: "2fa91f49-c3e5-497a-adae-535b1fa9b7fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:54.298049 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.297912    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2fa91f49-c3e5-497a-adae-535b1fa9b7fb" (UID: "2fa91f49-c3e5-497a-adae-535b1fa9b7fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:54.298049 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.298000    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-home" (OuterVolumeSpecName: "home") pod "2fa91f49-c3e5-497a-adae-535b1fa9b7fb" (UID: "2fa91f49-c3e5-497a-adae-535b1fa9b7fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:54.298163 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.298106    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:58:54.298163 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.298123    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:58:54.298163 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.298144    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:58:54.300077 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.300051    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-dshm" (OuterVolumeSpecName: "dshm") pod "2fa91f49-c3e5-497a-adae-535b1fa9b7fb" (UID: "2fa91f49-c3e5-497a-adae-535b1fa9b7fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:54.300077 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.300060    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-kube-api-access-6mvhc" (OuterVolumeSpecName: "kube-api-access-6mvhc") pod "2fa91f49-c3e5-497a-adae-535b1fa9b7fb" (UID: "2fa91f49-c3e5-497a-adae-535b1fa9b7fb"). InnerVolumeSpecName "kube-api-access-6mvhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:58:54.300077 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.300067    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fa91f49-c3e5-497a-adae-535b1fa9b7fb" (UID: "2fa91f49-c3e5-497a-adae-535b1fa9b7fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:58:54.398845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.398815    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:58:54.398845 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.398841    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:58:54.399003 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.398852    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mvhc\" (UniqueName: \"kubernetes.io/projected/2fa91f49-c3e5-497a-adae-535b1fa9b7fb-kube-api-access-6mvhc\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 13:58:54.834645 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.834610    2579 generic.go:356] "Generic (PLEG): container finished" podID="2fa91f49-c3e5-497a-adae-535b1fa9b7fb" containerID="f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f" exitCode=0
Jul 18 13:58:54.834825 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.834655    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"
Jul 18 13:58:54.834825 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.834704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf" event={"ID":"2fa91f49-c3e5-497a-adae-535b1fa9b7fb","Type":"ContainerDied","Data":"f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f"}
Jul 18 13:58:54.834825 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.834757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf" event={"ID":"2fa91f49-c3e5-497a-adae-535b1fa9b7fb","Type":"ContainerDied","Data":"6b52ecc037f65dfe422adfc885651289ba4ebb77139961706ddb24ed3ee6c5f9"}
Jul 18 13:58:54.834825 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.834783    2579 scope.go:117] "RemoveContainer" containerID="f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f"
Jul 18 13:58:54.844937 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.844919    2579 scope.go:117] "RemoveContainer" containerID="f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f"
Jul 18 13:58:54.845216 ip-10-0-135-180 kubenswrapper[2579]: E0718 13:58:54.845199    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f\": container with ID starting with f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f not found: ID does not exist" containerID="f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f"
Jul 18 13:58:54.845258 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.845224    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f"} err="failed to get container status \"f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f\": rpc error: code = NotFound desc = could not find container \"f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f\": container with ID starting with f100fb591dab5ea84b60dd429d3e15b80083de3fcf0dd5c6bc67597f98c66f5f not found: ID does not exist"
Jul 18 13:58:54.856148 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.856127    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"]
Jul 18 13:58:54.859424 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:54.859405    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-a9d318aa-kserve-58f9489dc5-h95vf"]
Jul 18 13:58:56.847807 ip-10-0-135-180 kubenswrapper[2579]: I0718 13:58:56.847775    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa91f49-c3e5-497a-adae-535b1fa9b7fb" path="/var/lib/kubelet/pods/2fa91f49-c3e5-497a-adae-535b1fa9b7fb/volumes"
Jul 18 14:07:35.191022 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.190991    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"]
Jul 18 14:07:35.191623 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.191336    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa91f49-c3e5-497a-adae-535b1fa9b7fb" containerName="main"
Jul 18 14:07:35.191623 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.191347    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa91f49-c3e5-497a-adae-535b1fa9b7fb" containerName="main"
Jul 18 14:07:35.191623 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.191431    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa91f49-c3e5-497a-adae-535b1fa9b7fb" containerName="main"
Jul 18 14:07:35.194539 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.194518    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.197892 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.197868    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffbfeb3b\"/\"default-dockercfg-ck2l9\""
Jul 18 14:07:35.197892 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.197882    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffbfeb3b\"/\"e2e-lifecycle-model-ffbfeb3b-kserve-self-signed-certs\""
Jul 18 14:07:35.198104 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.197919    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffbfeb3b\"/\"openshift-service-ca.crt\""
Jul 18 14:07:35.198104 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.197886    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffbfeb3b\"/\"kube-root-ca.crt\""
Jul 18 14:07:35.209736 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.209712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"]
Jul 18 14:07:35.262852 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.262823    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-dshm\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.262990 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.262875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tls-certs\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.262990 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.262920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkcxz\" (UniqueName: \"kubernetes.io/projected/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-kube-api-access-fkcxz\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.263060 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.262998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-home\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.263060 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.263017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-model-cache\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.263060 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.263035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tmp-dir\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.363867 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.363838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcxz\" (UniqueName: \"kubernetes.io/projected/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-kube-api-access-fkcxz\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364031 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.363892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-home\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364099 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-model-cache\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364099 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tmp-dir\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364232 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-dshm\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364232 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-home\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364232 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tls-certs\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364381 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tmp-dir\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.364442 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.364411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-model-cache\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.366306 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.366279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-dshm\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.366548 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.366525    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tls-certs\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.372316 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.372294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkcxz\" (UniqueName: \"kubernetes.io/projected/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-kube-api-access-fkcxz\") pod \"e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") " pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.503946 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.503914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:35.637895 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:35.637864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"]
Jul 18 14:07:35.641828 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:07:35.641801    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f93fb7d_ddf3_4e3f_b989_4f1648b6b7a2.slice/crio-0f39d222fc4ea9215fffd79899dd06217518209dc6fe48973df9b89cd6a551f9 WatchSource:0}: Error finding container 0f39d222fc4ea9215fffd79899dd06217518209dc6fe48973df9b89cd6a551f9: Status 404 returned error can't find the container with id 0f39d222fc4ea9215fffd79899dd06217518209dc6fe48973df9b89cd6a551f9
Jul 18 14:07:36.250260 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:36.250224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp" event={"ID":"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2","Type":"ContainerStarted","Data":"8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79"}
Jul 18 14:07:36.250622 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:36.250267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp" event={"ID":"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2","Type":"ContainerStarted","Data":"0f39d222fc4ea9215fffd79899dd06217518209dc6fe48973df9b89cd6a551f9"}
Jul 18 14:07:36.265781 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:36.265728    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp" podStartSLOduration=1.265713629 podStartE2EDuration="1.265713629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:07:36.264521936 +0000 UTC m=+2123.933971641" watchObservedRunningTime="2026-07-18 14:07:36.265713629 +0000 UTC m=+2123.935163334"
Jul 18 14:07:45.504937 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:45.504906    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:45.505427 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:45.505040    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:45.517348 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:45.517322    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:07:46.305447 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:07:46.305416    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:08:21.489528 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:21.489489    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"]
Jul 18 14:08:36.509809 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.509771    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp" podUID="3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" containerName="main" containerID="cri-o://8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79" gracePeriod=45
Jul 18 14:08:36.762525 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.762469    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:08:36.911600 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tmp-dir\") pod \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") "
Jul 18 14:08:36.911778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911641    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-home\") pod \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") "
Jul 18 14:08:36.911778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911686    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-model-cache\") pod \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") "
Jul 18 14:08:36.911778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911718    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-dshm\") pod \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") "
Jul 18 14:08:36.911778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911733    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tls-certs\") pod \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") "
Jul 18 14:08:36.911778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911765    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkcxz\" (UniqueName: \"kubernetes.io/projected/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-kube-api-access-fkcxz\") pod \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\" (UID: \"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2\") "
Jul 18 14:08:36.912014 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911897    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" (UID: "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:08:36.912014 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.911927    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-home" (OuterVolumeSpecName: "home") pod "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" (UID: "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:08:36.912118 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.912019    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" (UID: "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:08:36.912118 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.912040    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:08:36.912118 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.912057    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:08:36.914018 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.913993    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-dshm" (OuterVolumeSpecName: "dshm") pod "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" (UID: "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:08:36.914128 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.913997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" (UID: "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:08:36.914128 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:36.914033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-kube-api-access-fkcxz" (OuterVolumeSpecName: "kube-api-access-fkcxz") pod "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" (UID: "3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2"). InnerVolumeSpecName "kube-api-access-fkcxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:08:37.013271 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.013191    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:08:37.013271 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.013220    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:08:37.013271 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.013231    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:08:37.013271 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.013243    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkcxz\" (UniqueName: \"kubernetes.io/projected/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2-kube-api-access-fkcxz\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:08:37.533368 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.533324    2579 generic.go:356] "Generic (PLEG): container finished" podID="3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" containerID="8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79" exitCode=0
Jul 18 14:08:37.533868 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.533382    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"
Jul 18 14:08:37.533868 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.533411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp" event={"ID":"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2","Type":"ContainerDied","Data":"8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79"}
Jul 18 14:08:37.533868 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.533447    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp" event={"ID":"3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2","Type":"ContainerDied","Data":"0f39d222fc4ea9215fffd79899dd06217518209dc6fe48973df9b89cd6a551f9"}
Jul 18 14:08:37.533868 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.533463    2579 scope.go:117] "RemoveContainer" containerID="8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79"
Jul 18 14:08:37.542257 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.542236    2579 scope.go:117] "RemoveContainer" containerID="8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79"
Jul 18 14:08:37.542563 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:08:37.542534    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79\": container with ID starting with 8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79 not found: ID does not exist" containerID="8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79"
Jul 18 14:08:37.542695 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.542570    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79"} err="failed to get container status \"8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79\": rpc error: code = NotFound desc = could not find container \"8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79\": container with ID starting with 8d81d0e19bebfd8279ccdef5d6bf8872457fb582f303e15f48fe89b8c4751f79 not found: ID does not exist"
Jul 18 14:08:37.556017 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.555992    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"]
Jul 18 14:08:37.559973 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:37.559953    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ffbfeb3b/e2e-lifecycle-model-ffbfeb3b-kserve-856b65dbf9-57gxp"]
Jul 18 14:08:38.847068 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:08:38.847036    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" path="/var/lib/kubelet/pods/3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2/volumes"
Jul 18 14:10:24.187253 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.187221    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"]
Jul 18 14:10:24.187739 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.187604    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" containerName="main"
Jul 18 14:10:24.187739 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.187615    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" containerName="main"
Jul 18 14:10:24.187739 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.187690    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f93fb7d-ddf3-4e3f-b989-4f1648b6b7a2" containerName="main"
Jul 18 14:10:24.190141 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.190125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.193459 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.193435    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e5f2a1\"/\"kube-root-ca.crt\""
Jul 18 14:10:24.193459 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.193452    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e5f2a1\"/\"default-dockercfg-2j47w\""
Jul 18 14:10:24.193636 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.193437    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e5f2a1\"/\"test-model-e5f2a1-kserve-self-signed-certs\""
Jul 18 14:10:24.193636 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.193483    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e5f2a1\"/\"openshift-service-ca.crt\""
Jul 18 14:10:24.199249 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.199225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"]
Jul 18 14:10:24.354798 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.354765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-home\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.354949 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.354806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tls-certs\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.354949 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.354894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqplf\" (UniqueName: \"kubernetes.io/projected/cfb3bf7a-0c28-4db3-952b-beb895bb94db-kube-api-access-vqplf\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.354949 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.354924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-dshm\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.355055 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.354957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tmp-dir\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.355055 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.355042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-model-cache\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.455702 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.455595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-home\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.455702 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.455639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tls-certs\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.455927 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.455731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqplf\" (UniqueName: \"kubernetes.io/projected/cfb3bf7a-0c28-4db3-952b-beb895bb94db-kube-api-access-vqplf\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.455927 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.455829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-dshm\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.455927 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.455862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tmp-dir\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.456072 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.456004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-model-cache\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.456072 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.456012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-home\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.456189 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.456173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tmp-dir\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.456285 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.456258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-model-cache\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.458023 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.457993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-dshm\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.458253 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.458237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tls-certs\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.462748 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.462731    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqplf\" (UniqueName: \"kubernetes.io/projected/cfb3bf7a-0c28-4db3-952b-beb895bb94db-kube-api-access-vqplf\") pod \"test-model-e5f2a1-kserve-7688875fb9-d26kz\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") " pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.499764 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.499743    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:24.627778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:24.627755    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"]
Jul 18 14:10:24.630183 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:10:24.630155    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfb3bf7a_0c28_4db3_952b_beb895bb94db.slice/crio-c95e32b9a748fc8640d31f71d928c406d0f278484950ef7d228d773062a3edc0 WatchSource:0}: Error finding container c95e32b9a748fc8640d31f71d928c406d0f278484950ef7d228d773062a3edc0: Status 404 returned error can't find the container with id c95e32b9a748fc8640d31f71d928c406d0f278484950ef7d228d773062a3edc0
Jul 18 14:10:25.057402 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:25.057367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz" event={"ID":"cfb3bf7a-0c28-4db3-952b-beb895bb94db","Type":"ContainerStarted","Data":"f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3"}
Jul 18 14:10:25.057556 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:25.057413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz" event={"ID":"cfb3bf7a-0c28-4db3-952b-beb895bb94db","Type":"ContainerStarted","Data":"c95e32b9a748fc8640d31f71d928c406d0f278484950ef7d228d773062a3edc0"}
Jul 18 14:10:25.074934 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:25.074885    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz" podStartSLOduration=1.074868721 podStartE2EDuration="1.074868721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:10:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:10:25.072737576 +0000 UTC m=+2292.742187280" watchObservedRunningTime="2026-07-18 14:10:25.074868721 +0000 UTC m=+2292.744318428"
Jul 18 14:10:34.500092 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:34.500042    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:34.500573 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:34.500152    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:34.512412 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:34.512392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:35.118495 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:35.118469    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:10:49.692972 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.692937    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"]
Jul 18 14:10:49.696198 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.696180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.698801 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.698779    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-45d555\"/\"kube-root-ca.crt\""
Jul 18 14:10:49.699563 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.699543    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-45d555\"/\"openshift-service-ca.crt\""
Jul 18 14:10:49.699652 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.699546    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-45d555\"/\"default-dockercfg-2f458\""
Jul 18 14:10:49.699723 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.699550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-45d555\"/\"test-model-45d555-kserve-self-signed-certs\""
Jul 18 14:10:49.705879 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.705837    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"]
Jul 18 14:10:49.781047 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.781014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69gl8\" (UniqueName: \"kubernetes.io/projected/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-kube-api-access-69gl8\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.781198 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.781066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tmp-dir\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.781198 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.781092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-dshm\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.781198 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.781128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tls-certs\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.781198 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.781151    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-home\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.781198 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.781185    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-model-cache\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882057 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-dshm\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882246 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tls-certs\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882246 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-home\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882246 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-model-cache\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882246 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69gl8\" (UniqueName: \"kubernetes.io/projected/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-kube-api-access-69gl8\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882426 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tmp-dir\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882620 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-model-cache\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882784 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882724    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-home\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.882858 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.882841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tmp-dir\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.884369 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.884348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-dshm\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.884760 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.884739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tls-certs\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:49.889285 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:49.889265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69gl8\" (UniqueName: \"kubernetes.io/projected/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-kube-api-access-69gl8\") pod \"test-model-45d555-kserve-b88dfc6df-htfw8\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") " pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:50.006704 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:50.006647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:10:50.139177 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:50.139093    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"]
Jul 18 14:10:50.141950 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:10:50.141921    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d8be8eb_9d36_4ccd_a9c0_fd257689df8a.slice/crio-e9c01d6e5f594cfaaaf547a9c414a11af3a5b34fa576faefe46403cd9cbc01a7 WatchSource:0}: Error finding container e9c01d6e5f594cfaaaf547a9c414a11af3a5b34fa576faefe46403cd9cbc01a7: Status 404 returned error can't find the container with id e9c01d6e5f594cfaaaf547a9c414a11af3a5b34fa576faefe46403cd9cbc01a7
Jul 18 14:10:50.179097 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:50.179029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8" event={"ID":"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a","Type":"ContainerStarted","Data":"e9c01d6e5f594cfaaaf547a9c414a11af3a5b34fa576faefe46403cd9cbc01a7"}
Jul 18 14:10:51.185415 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:51.185376    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8" event={"ID":"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a","Type":"ContainerStarted","Data":"923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb"}
Jul 18 14:10:51.200727 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:10:51.200660    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8" podStartSLOduration=2.200644745 podStartE2EDuration="2.200644745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:10:51.199358153 +0000 UTC m=+2318.868807859" watchObservedRunningTime="2026-07-18 14:10:51.200644745 +0000 UTC m=+2318.870094449"
Jul 18 14:11:00.007462 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:00.007421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:11:00.007967 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:00.007496    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:11:00.019744 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:00.019725    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:11:00.244894 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:00.244862    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:11:07.028659 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:07.028620    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"]
Jul 18 14:11:08.018116 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:08.018067    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"]
Jul 18 14:11:22.050893 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.050844    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz" podUID="cfb3bf7a-0c28-4db3-952b-beb895bb94db" containerName="main" containerID="cri-o://f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3" gracePeriod=45
Jul 18 14:11:22.290130 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.290105    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:11:22.347475 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.347399    2579 generic.go:356] "Generic (PLEG): container finished" podID="cfb3bf7a-0c28-4db3-952b-beb895bb94db" containerID="f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3" exitCode=0
Jul 18 14:11:22.347475 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.347449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"
Jul 18 14:11:22.347626 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.347449    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz" event={"ID":"cfb3bf7a-0c28-4db3-952b-beb895bb94db","Type":"ContainerDied","Data":"f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3"}
Jul 18 14:11:22.347626 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.347569    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz" event={"ID":"cfb3bf7a-0c28-4db3-952b-beb895bb94db","Type":"ContainerDied","Data":"c95e32b9a748fc8640d31f71d928c406d0f278484950ef7d228d773062a3edc0"}
Jul 18 14:11:22.347626 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.347597    2579 scope.go:117] "RemoveContainer" containerID="f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3"
Jul 18 14:11:22.355718 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.355699    2579 scope.go:117] "RemoveContainer" containerID="f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3"
Jul 18 14:11:22.355960 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:11:22.355941    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3\": container with ID starting with f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3 not found: ID does not exist" containerID="f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3"
Jul 18 14:11:22.356012 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.355969    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3"} err="failed to get container status \"f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3\": rpc error: code = NotFound desc = could not find container \"f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3\": container with ID starting with f1c5b90e8b31cbf3705abe96213f5626ab0f54eb5dcff4347381f5e3298d2df3 not found: ID does not exist"
Jul 18 14:11:22.387537 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387517    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-home\") pod \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") "
Jul 18 14:11:22.387644 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387564    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqplf\" (UniqueName: \"kubernetes.io/projected/cfb3bf7a-0c28-4db3-952b-beb895bb94db-kube-api-access-vqplf\") pod \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") "
Jul 18 14:11:22.387644 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387591    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tmp-dir\") pod \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") "
Jul 18 14:11:22.387644 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387611    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tls-certs\") pod \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") "
Jul 18 14:11:22.387644 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387644    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-model-cache\") pod \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") "
Jul 18 14:11:22.387874 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387806    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-home" (OuterVolumeSpecName: "home") pod "cfb3bf7a-0c28-4db3-952b-beb895bb94db" (UID: "cfb3bf7a-0c28-4db3-952b-beb895bb94db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:22.387874 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387816    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-dshm\") pod \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\" (UID: \"cfb3bf7a-0c28-4db3-952b-beb895bb94db\") "
Jul 18 14:11:22.387974 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387906    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfb3bf7a-0c28-4db3-952b-beb895bb94db" (UID: "cfb3bf7a-0c28-4db3-952b-beb895bb94db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:22.387974 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.387923    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfb3bf7a-0c28-4db3-952b-beb895bb94db" (UID: "cfb3bf7a-0c28-4db3-952b-beb895bb94db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:22.388160 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.388143    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:22.388210 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.388165    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:22.388210 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.388178    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:22.389813 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.389785    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-dshm" (OuterVolumeSpecName: "dshm") pod "cfb3bf7a-0c28-4db3-952b-beb895bb94db" (UID: "cfb3bf7a-0c28-4db3-952b-beb895bb94db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:22.389904 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.389864    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfb3bf7a-0c28-4db3-952b-beb895bb94db" (UID: "cfb3bf7a-0c28-4db3-952b-beb895bb94db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:11:22.389964 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.389946    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb3bf7a-0c28-4db3-952b-beb895bb94db-kube-api-access-vqplf" (OuterVolumeSpecName: "kube-api-access-vqplf") pod "cfb3bf7a-0c28-4db3-952b-beb895bb94db" (UID: "cfb3bf7a-0c28-4db3-952b-beb895bb94db"). InnerVolumeSpecName "kube-api-access-vqplf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:11:22.489479 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.489453    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqplf\" (UniqueName: \"kubernetes.io/projected/cfb3bf7a-0c28-4db3-952b-beb895bb94db-kube-api-access-vqplf\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:22.489479 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.489475    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb3bf7a-0c28-4db3-952b-beb895bb94db-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:22.489479 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.489484    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb3bf7a-0c28-4db3-952b-beb895bb94db-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:22.668342 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.668309    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"]
Jul 18 14:11:22.671759 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.671736    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e5f2a1/test-model-e5f2a1-kserve-7688875fb9-d26kz"]
Jul 18 14:11:22.847144 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:22.847114    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfb3bf7a-0c28-4db3-952b-beb895bb94db" path="/var/lib/kubelet/pods/cfb3bf7a-0c28-4db3-952b-beb895bb94db/volumes"
Jul 18 14:11:23.042062 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.042019    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8" podUID="3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" containerName="main" containerID="cri-o://923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb" gracePeriod=45
Jul 18 14:11:23.290310 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.290286    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:11:23.353926 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.353835    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" containerID="923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb" exitCode=0
Jul 18 14:11:23.353926 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.353879    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"
Jul 18 14:11:23.353926 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.353891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8" event={"ID":"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a","Type":"ContainerDied","Data":"923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb"}
Jul 18 14:11:23.353926 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.353923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8" event={"ID":"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a","Type":"ContainerDied","Data":"e9c01d6e5f594cfaaaf547a9c414a11af3a5b34fa576faefe46403cd9cbc01a7"}
Jul 18 14:11:23.354189 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.353943    2579 scope.go:117] "RemoveContainer" containerID="923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb"
Jul 18 14:11:23.363146 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.363131    2579 scope.go:117] "RemoveContainer" containerID="923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb"
Jul 18 14:11:23.363373 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:11:23.363355    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb\": container with ID starting with 923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb not found: ID does not exist" containerID="923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb"
Jul 18 14:11:23.363431 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.363382    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb"} err="failed to get container status \"923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb\": rpc error: code = NotFound desc = could not find container \"923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb\": container with ID starting with 923946b5dc2769e3a56debf1f1cbd178b2276687bccf00d3f80e4769532627cb not found: ID does not exist"
Jul 18 14:11:23.396735 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.396713    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69gl8\" (UniqueName: \"kubernetes.io/projected/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-kube-api-access-69gl8\") pod \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") "
Jul 18 14:11:23.396854 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.396759    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tls-certs\") pod \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") "
Jul 18 14:11:23.396854 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.396811    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-model-cache\") pod \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") "
Jul 18 14:11:23.396854 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.396842    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tmp-dir\") pod \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") "
Jul 18 14:11:23.396964 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.396892    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-home\") pod \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") "
Jul 18 14:11:23.396964 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.396913    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-dshm\") pod \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\" (UID: \"3d8be8eb-9d36-4ccd-a9c0-fd257689df8a\") "
Jul 18 14:11:23.397161 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.397108    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" (UID: "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:23.397161 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.397155    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" (UID: "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:23.397299 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.397159    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-home" (OuterVolumeSpecName: "home") pod "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" (UID: "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:23.398924 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.398900    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-dshm" (OuterVolumeSpecName: "dshm") pod "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" (UID: "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:23.399048 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.399030    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" (UID: "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:11:23.399091 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.399077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-kube-api-access-69gl8" (OuterVolumeSpecName: "kube-api-access-69gl8") pod "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" (UID: "3d8be8eb-9d36-4ccd-a9c0-fd257689df8a"). InnerVolumeSpecName "kube-api-access-69gl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:11:23.498201 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.498171    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:23.498201 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.498197    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:23.498201 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.498206    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:23.498381 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.498214    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69gl8\" (UniqueName: \"kubernetes.io/projected/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-kube-api-access-69gl8\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:23.498381 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.498222    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:23.498381 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.498230    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:11:23.674718 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.674689    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"]
Jul 18 14:11:23.680644 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:23.680621    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-45d555/test-model-45d555-kserve-b88dfc6df-htfw8"]
Jul 18 14:11:24.847530 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:11:24.847498    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" path="/var/lib/kubelet/pods/3d8be8eb-9d36-4ccd-a9c0-fd257689df8a/volumes"
Jul 18 14:13:49.771270 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771233    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"]
Jul 18 14:13:49.771807 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771791    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" containerName="main"
Jul 18 14:13:49.771888 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771811    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" containerName="main"
Jul 18 14:13:49.771888 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771826    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfb3bf7a-0c28-4db3-952b-beb895bb94db" containerName="main"
Jul 18 14:13:49.771888 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771834    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfb3bf7a-0c28-4db3-952b-beb895bb94db" containerName="main"
Jul 18 14:13:49.772044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771941    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d8be8eb-9d36-4ccd-a9c0-fd257689df8a" containerName="main"
Jul 18 14:13:49.772044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.771955    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfb3bf7a-0c28-4db3-952b-beb895bb94db" containerName="main"
Jul 18 14:13:49.774807 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.774785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.778109 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.778085    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1bdd6b\"/\"default-dockercfg-5cn54\""
Jul 18 14:13:49.778233 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.778137    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1bdd6b\"/\"openshift-service-ca.crt\""
Jul 18 14:13:49.778233 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.778147    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1bdd6b\"/\"kube-root-ca.crt\""
Jul 18 14:13:49.778233 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.778137    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1bdd6b\"/\"auth-test-model-1bdd6b-kserve-self-signed-certs\""
Jul 18 14:13:49.782794 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.782747    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"]
Jul 18 14:13:49.891044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.891007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-model-cache\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.891228 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.891114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4pcz\" (UniqueName: \"kubernetes.io/projected/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-kube-api-access-j4pcz\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.891228 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.891147    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tls-certs\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.891228 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.891209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-dshm\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.891357 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.891246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-home\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.891357 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.891272    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tmp-dir\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992374 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4pcz\" (UniqueName: \"kubernetes.io/projected/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-kube-api-access-j4pcz\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992374 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tls-certs\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992583 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-dshm\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992583 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-home\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992583 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tmp-dir\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992583 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992536    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-model-cache\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.992948 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-home\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.993045 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tmp-dir\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.993045 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.992984    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-model-cache\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.994849 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.994825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-dshm\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.994973 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.994958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tls-certs\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:49.999735 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:49.999709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4pcz\" (UniqueName: \"kubernetes.io/projected/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-kube-api-access-j4pcz\") pod \"auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") " pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:50.085738 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:50.085639    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:13:50.223195 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:50.223168    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"]
Jul 18 14:13:50.226022 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:13:50.225984    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a69cf8e_31ec_4464_b441_dbd3cbdec5d1.slice/crio-e6d32a7a3ff0fd750f4f1fced95dbd25fc755f92971a2704be0f2cb4c57437e0 WatchSource:0}: Error finding container e6d32a7a3ff0fd750f4f1fced95dbd25fc755f92971a2704be0f2cb4c57437e0: Status 404 returned error can't find the container with id e6d32a7a3ff0fd750f4f1fced95dbd25fc755f92971a2704be0f2cb4c57437e0
Jul 18 14:13:51.045105 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:51.045067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc" event={"ID":"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1","Type":"ContainerStarted","Data":"a63dad39727283d2be390778eb10e2a10dadab4fec872a294f51a22c3ef6b802"}
Jul 18 14:13:51.045487 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:51.045116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc" event={"ID":"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1","Type":"ContainerStarted","Data":"e6d32a7a3ff0fd750f4f1fced95dbd25fc755f92971a2704be0f2cb4c57437e0"}
Jul 18 14:13:51.061697 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:13:51.061635    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc" podStartSLOduration=2.061621603 podStartE2EDuration="2.061621603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:13:51.059739172 +0000 UTC m=+2498.729188877" watchObservedRunningTime="2026-07-18 14:13:51.061621603 +0000 UTC m=+2498.731071307"
Jul 18 14:14:00.086223 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:00.086190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:14:00.086720 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:00.086240    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:14:00.098391 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:00.098367    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:14:01.102572 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:01.102543    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:14:23.682471 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.682432    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"]
Jul 18 14:14:23.686033 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.686014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.689076 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.689050    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"auth-test-model-66d5d1-kserve-self-signed-certs\""
Jul 18 14:14:23.689173 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.689053    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"openshift-service-ca.crt\""
Jul 18 14:14:23.689173 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.689124    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"default-dockercfg-xnxfp\""
Jul 18 14:14:23.689173 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.689143    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"kube-root-ca.crt\""
Jul 18 14:14:23.695472 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.695451    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"]
Jul 18 14:14:23.784100 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.784078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tmp-dir\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.784217 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.784107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqf6j\" (UniqueName: \"kubernetes.io/projected/487b9d47-0ca9-41ad-97e1-edeae18db0c3-kube-api-access-gqf6j\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.784217 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.784129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tls-certs\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.784217 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.784206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-dshm\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.784324 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.784267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-model-cache\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.784324 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.784297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-home\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885324 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-dshm\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885475 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-model-cache\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885475 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-home\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885475 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tmp-dir\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885475 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqf6j\" (UniqueName: \"kubernetes.io/projected/487b9d47-0ca9-41ad-97e1-edeae18db0c3-kube-api-access-gqf6j\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885712 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tls-certs\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885857 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tmp-dir\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885925 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-model-cache\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.885925 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.885878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-home\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.887505 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.887485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-dshm\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.887859 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.887839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tls-certs\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.895003 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.894979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqf6j\" (UniqueName: \"kubernetes.io/projected/487b9d47-0ca9-41ad-97e1-edeae18db0c3-kube-api-access-gqf6j\") pod \"auth-test-model-66d5d1-kserve-76669ffb59-gnhw5\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") " pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:23.996338 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:23.996312    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:24.124326 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:24.124303    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"]
Jul 18 14:14:24.126415 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:14:24.126384    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod487b9d47_0ca9_41ad_97e1_edeae18db0c3.slice/crio-da3a89792b8839e0a84498c1a69573da62c6d12f689e75aafa303a80274ff79f WatchSource:0}: Error finding container da3a89792b8839e0a84498c1a69573da62c6d12f689e75aafa303a80274ff79f: Status 404 returned error can't find the container with id da3a89792b8839e0a84498c1a69573da62c6d12f689e75aafa303a80274ff79f
Jul 18 14:14:24.203928 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:24.203885    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5" event={"ID":"487b9d47-0ca9-41ad-97e1-edeae18db0c3","Type":"ContainerStarted","Data":"26bb45a13fcf450c40a835c31ba39c1a729dab90414e34b084c0241e34bfa83c"}
Jul 18 14:14:24.204078 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:24.203941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5" event={"ID":"487b9d47-0ca9-41ad-97e1-edeae18db0c3","Type":"ContainerStarted","Data":"da3a89792b8839e0a84498c1a69573da62c6d12f689e75aafa303a80274ff79f"}
Jul 18 14:14:24.224664 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:24.224621    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5" podStartSLOduration=1.224605032 podStartE2EDuration="1.224605032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:14:24.224561465 +0000 UTC m=+2531.894011169" watchObservedRunningTime="2026-07-18 14:14:24.224605032 +0000 UTC m=+2531.894054734"
Jul 18 14:14:33.997255 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:33.997213    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:33.997715 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:33.997272    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:34.009768 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:34.009743    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:14:34.264260 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:14:34.264177    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:16:23.598761 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:23.598724    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"]
Jul 18 14:16:24.603753 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:24.603722    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"]
Jul 18 14:16:34.492595 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.492560    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"]
Jul 18 14:16:34.496481 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.496462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.498620 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.498603    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1bdd6b\"/\"sub-test-model-1bdd6b-kserve-self-signed-certs\""
Jul 18 14:16:34.502604 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.502576    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"]
Jul 18 14:16:34.562067 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.562033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4127e7-0692-4ace-814c-7e2c2fbbc553-tls-certs\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.562067 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.562071    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxnrh\" (UniqueName: \"kubernetes.io/projected/de4127e7-0692-4ace-814c-7e2c2fbbc553-kube-api-access-mxnrh\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.562307 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.562100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-home\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.562307 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.562238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-model-cache\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.562307 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.562295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-tmp-dir\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.562451 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.562325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-dshm\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663206 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4127e7-0692-4ace-814c-7e2c2fbbc553-tls-certs\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663206 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnrh\" (UniqueName: \"kubernetes.io/projected/de4127e7-0692-4ace-814c-7e2c2fbbc553-kube-api-access-mxnrh\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663438 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-home\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663506 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-model-cache\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663569 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-tmp-dir\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663621 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-dshm\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663740 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-home\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663872 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-model-cache\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.663949 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.663874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-tmp-dir\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.665711 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.665689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-dshm\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.665790 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.665733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4127e7-0692-4ace-814c-7e2c2fbbc553-tls-certs\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.671152 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.671132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxnrh\" (UniqueName: \"kubernetes.io/projected/de4127e7-0692-4ace-814c-7e2c2fbbc553-kube-api-access-mxnrh\") pod \"sub-test-model-1bdd6b-kserve-6c48c7b549-q252h\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") " pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.806923 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.806842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:34.942776 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:34.942747    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"]
Jul 18 14:16:34.944904 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:16:34.944878    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde4127e7_0692_4ace_814c_7e2c2fbbc553.slice/crio-966f2520ddc4419134c7e377d7199bc06f4795cb3583ff0777f53e0d10f7a6fc WatchSource:0}: Error finding container 966f2520ddc4419134c7e377d7199bc06f4795cb3583ff0777f53e0d10f7a6fc: Status 404 returned error can't find the container with id 966f2520ddc4419134c7e377d7199bc06f4795cb3583ff0777f53e0d10f7a6fc
Jul 18 14:16:35.824577 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:35.824540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h" event={"ID":"de4127e7-0692-4ace-814c-7e2c2fbbc553","Type":"ContainerStarted","Data":"f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2"}
Jul 18 14:16:35.824577 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:35.824593    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h" event={"ID":"de4127e7-0692-4ace-814c-7e2c2fbbc553","Type":"ContainerStarted","Data":"966f2520ddc4419134c7e377d7199bc06f4795cb3583ff0777f53e0d10f7a6fc"}
Jul 18 14:16:35.842096 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:35.842051    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h" podStartSLOduration=1.842033678 podStartE2EDuration="1.842033678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:16:35.839337563 +0000 UTC m=+2663.508787268" watchObservedRunningTime="2026-07-18 14:16:35.842033678 +0000 UTC m=+2663.511483384"
Jul 18 14:16:38.619007 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.618968    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc" podUID="5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" containerName="main" containerID="cri-o://a63dad39727283d2be390778eb10e2a10dadab4fec872a294f51a22c3ef6b802" gracePeriod=45
Jul 18 14:16:38.843220 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.843189    2579 generic.go:356] "Generic (PLEG): container finished" podID="5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" containerID="a63dad39727283d2be390778eb10e2a10dadab4fec872a294f51a22c3ef6b802" exitCode=0
Jul 18 14:16:38.847566 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.847543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc" event={"ID":"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1","Type":"ContainerDied","Data":"a63dad39727283d2be390778eb10e2a10dadab4fec872a294f51a22c3ef6b802"}
Jul 18 14:16:38.862260 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.862242    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:16:38.897318 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897256    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tmp-dir\") pod \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") "
Jul 18 14:16:38.897453 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897329    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-dshm\") pod \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") "
Jul 18 14:16:38.897453 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897366    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tls-certs\") pod \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") "
Jul 18 14:16:38.897453 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897421    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4pcz\" (UniqueName: \"kubernetes.io/projected/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-kube-api-access-j4pcz\") pod \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") "
Jul 18 14:16:38.897453 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897448    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-model-cache\") pod \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") "
Jul 18 14:16:38.897663 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897474    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-home\") pod \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\" (UID: \"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1\") "
Jul 18 14:16:38.897663 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897523    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" (UID: "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:38.897783 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897761    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" (UID: "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:38.897887 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897860    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-home" (OuterVolumeSpecName: "home") pod "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" (UID: "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:38.897953 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897891    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:38.897953 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.897911    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:38.899562 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.899538    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-dshm" (OuterVolumeSpecName: "dshm") pod "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" (UID: "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:38.899706 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.899604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-kube-api-access-j4pcz" (OuterVolumeSpecName: "kube-api-access-j4pcz") pod "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" (UID: "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1"). InnerVolumeSpecName "kube-api-access-j4pcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:16:38.899925 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.899904    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" (UID: "5a69cf8e-31ec-4464-b441-dbd3cbdec5d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:16:38.999106 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.999080    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:38.999106 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.999104    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:38.999282 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.999114    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4pcz\" (UniqueName: \"kubernetes.io/projected/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-kube-api-access-j4pcz\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:38.999282 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:38.999124    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:39.623951 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.623912    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5" podUID="487b9d47-0ca9-41ad-97e1-edeae18db0c3" containerName="main" containerID="cri-o://26bb45a13fcf450c40a835c31ba39c1a729dab90414e34b084c0241e34bfa83c" gracePeriod=45
Jul 18 14:16:39.850199 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.850173    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"
Jul 18 14:16:39.850373 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.850171    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc" event={"ID":"5a69cf8e-31ec-4464-b441-dbd3cbdec5d1","Type":"ContainerDied","Data":"e6d32a7a3ff0fd750f4f1fced95dbd25fc755f92971a2704be0f2cb4c57437e0"}
Jul 18 14:16:39.850373 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.850305    2579 scope.go:117] "RemoveContainer" containerID="a63dad39727283d2be390778eb10e2a10dadab4fec872a294f51a22c3ef6b802"
Jul 18 14:16:39.851926 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.851899    2579 generic.go:356] "Generic (PLEG): container finished" podID="487b9d47-0ca9-41ad-97e1-edeae18db0c3" containerID="26bb45a13fcf450c40a835c31ba39c1a729dab90414e34b084c0241e34bfa83c" exitCode=0
Jul 18 14:16:39.852050 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.851974    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5" event={"ID":"487b9d47-0ca9-41ad-97e1-edeae18db0c3","Type":"ContainerDied","Data":"26bb45a13fcf450c40a835c31ba39c1a729dab90414e34b084c0241e34bfa83c"}
Jul 18 14:16:39.871070 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.871047    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"]
Jul 18 14:16:39.875763 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.875709    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/auth-test-model-1bdd6b-kserve-d7477bd79-xcwrc"]
Jul 18 14:16:39.884683 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:39.884654    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:16:40.008554 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008527    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqf6j\" (UniqueName: \"kubernetes.io/projected/487b9d47-0ca9-41ad-97e1-edeae18db0c3-kube-api-access-gqf6j\") pod \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") "
Jul 18 14:16:40.008704 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008564    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-dshm\") pod \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") "
Jul 18 14:16:40.008704 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008588    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-model-cache\") pod \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") "
Jul 18 14:16:40.008704 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008611    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-home\") pod \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") "
Jul 18 14:16:40.008704 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tls-certs\") pod \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") "
Jul 18 14:16:40.008704 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008660    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tmp-dir\") pod \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\" (UID: \"487b9d47-0ca9-41ad-97e1-edeae18db0c3\") "
Jul 18 14:16:40.008993 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008839    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "487b9d47-0ca9-41ad-97e1-edeae18db0c3" (UID: "487b9d47-0ca9-41ad-97e1-edeae18db0c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:40.008993 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-home" (OuterVolumeSpecName: "home") pod "487b9d47-0ca9-41ad-97e1-edeae18db0c3" (UID: "487b9d47-0ca9-41ad-97e1-edeae18db0c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:40.009093 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.008983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "487b9d47-0ca9-41ad-97e1-edeae18db0c3" (UID: "487b9d47-0ca9-41ad-97e1-edeae18db0c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:40.009156 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.009097    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:40.009156 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.009115    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:40.009156 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.009125    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:40.010819 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.010798    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "487b9d47-0ca9-41ad-97e1-edeae18db0c3" (UID: "487b9d47-0ca9-41ad-97e1-edeae18db0c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:16:40.010919 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.010844    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-dshm" (OuterVolumeSpecName: "dshm") pod "487b9d47-0ca9-41ad-97e1-edeae18db0c3" (UID: "487b9d47-0ca9-41ad-97e1-edeae18db0c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:16:40.010919 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.010849    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487b9d47-0ca9-41ad-97e1-edeae18db0c3-kube-api-access-gqf6j" (OuterVolumeSpecName: "kube-api-access-gqf6j") pod "487b9d47-0ca9-41ad-97e1-edeae18db0c3" (UID: "487b9d47-0ca9-41ad-97e1-edeae18db0c3"). InnerVolumeSpecName "kube-api-access-gqf6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:16:40.110405 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.110368    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqf6j\" (UniqueName: \"kubernetes.io/projected/487b9d47-0ca9-41ad-97e1-edeae18db0c3-kube-api-access-gqf6j\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:40.110405 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.110399    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/487b9d47-0ca9-41ad-97e1-edeae18db0c3-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:40.110405 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.110409    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/487b9d47-0ca9-41ad-97e1-edeae18db0c3-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:16:40.847935 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.847902    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" path="/var/lib/kubelet/pods/5a69cf8e-31ec-4464-b441-dbd3cbdec5d1/volumes"
Jul 18 14:16:40.858883 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.858854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5" event={"ID":"487b9d47-0ca9-41ad-97e1-edeae18db0c3","Type":"ContainerDied","Data":"da3a89792b8839e0a84498c1a69573da62c6d12f689e75aafa303a80274ff79f"}
Jul 18 14:16:40.858883 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.858873    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"
Jul 18 14:16:40.859053 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.858892    2579 scope.go:117] "RemoveContainer" containerID="26bb45a13fcf450c40a835c31ba39c1a729dab90414e34b084c0241e34bfa83c"
Jul 18 14:16:40.873772 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.873749    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"]
Jul 18 14:16:40.876950 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:40.876927    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/auth-test-model-66d5d1-kserve-76669ffb59-gnhw5"]
Jul 18 14:16:42.849237 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:42.849204    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="487b9d47-0ca9-41ad-97e1-edeae18db0c3" path="/var/lib/kubelet/pods/487b9d47-0ca9-41ad-97e1-edeae18db0c3/volumes"
Jul 18 14:16:44.807833 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:44.807798    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:44.808235 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:44.807855    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:44.820152 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:44.820125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:16:44.889596 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:16:44.889566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.995846    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"]
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.996741    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" containerName="main"
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.996761    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" containerName="main"
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.996781    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487b9d47-0ca9-41ad-97e1-edeae18db0c3" containerName="main"
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.996790    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="487b9d47-0ca9-41ad-97e1-edeae18db0c3" containerName="main"
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.996964    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="487b9d47-0ca9-41ad-97e1-edeae18db0c3" containerName="main"
Jul 18 14:17:08.000546 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:07.996984    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a69cf8e-31ec-4464-b441-dbd3cbdec5d1" containerName="main"
Jul 18 14:17:08.001365 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.001344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.005170 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.005043    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"kube-root-ca.crt\""
Jul 18 14:17:08.005170 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.005118    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"sub-test-model-66d5d1-kserve-self-signed-certs\""
Jul 18 14:17:08.005444 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.005262    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"openshift-service-ca.crt\""
Jul 18 14:17:08.005537 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.005456    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"default-dockercfg-xnxfp\""
Jul 18 14:17:08.008614 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.008590    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"]
Jul 18 14:17:08.167612 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.167573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-home\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.167804 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.167696    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-tmp-dir\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.167804 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.167764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-dshm\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.167916 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.167802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-model-cache\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.167916 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.167829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c87sw\" (UniqueName: \"kubernetes.io/projected/57610f77-7dde-47dd-adfc-c396737d0e94-kube-api-access-c87sw\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.167916 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.167852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57610f77-7dde-47dd-adfc-c396737d0e94-tls-certs\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.268722 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.268614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-home\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.268722 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.268689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-tmp-dir\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.268722 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.268723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-dshm\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.268995 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.268743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-model-cache\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.268995 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.268767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c87sw\" (UniqueName: \"kubernetes.io/projected/57610f77-7dde-47dd-adfc-c396737d0e94-kube-api-access-c87sw\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.269111 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.268995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57610f77-7dde-47dd-adfc-c396737d0e94-tls-certs\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.269111 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.269078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-home\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.269225 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.269109    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-tmp-dir\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.269281 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.269228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-model-cache\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.270964 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.270940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-dshm\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.271577 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.271555    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57610f77-7dde-47dd-adfc-c396737d0e94-tls-certs\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.276303 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.276280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c87sw\" (UniqueName: \"kubernetes.io/projected/57610f77-7dde-47dd-adfc-c396737d0e94-kube-api-access-c87sw\") pod \"sub-test-model-66d5d1-kserve-66cd8fd479-czjwb\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") " pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.313287 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.313262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:08.440615 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.440541    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"]
Jul 18 14:17:08.444085 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:17:08.444046    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57610f77_7dde_47dd_adfc_c396737d0e94.slice/crio-5fa1aa04edf942f9240c792baa23f90b5ee9ffeb39d9957c32527f5e1f20fc4b WatchSource:0}: Error finding container 5fa1aa04edf942f9240c792baa23f90b5ee9ffeb39d9957c32527f5e1f20fc4b: Status 404 returned error can't find the container with id 5fa1aa04edf942f9240c792baa23f90b5ee9ffeb39d9957c32527f5e1f20fc4b
Jul 18 14:17:08.993071 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.993038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb" event={"ID":"57610f77-7dde-47dd-adfc-c396737d0e94","Type":"ContainerStarted","Data":"e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727"}
Jul 18 14:17:08.993071 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:08.993076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb" event={"ID":"57610f77-7dde-47dd-adfc-c396737d0e94","Type":"ContainerStarted","Data":"5fa1aa04edf942f9240c792baa23f90b5ee9ffeb39d9957c32527f5e1f20fc4b"}
Jul 18 14:17:09.008960 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:09.008912    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb" podStartSLOduration=2.008895169 podStartE2EDuration="2.008895169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:17:09.007858258 +0000 UTC m=+2696.677307963" watchObservedRunningTime="2026-07-18 14:17:09.008895169 +0000 UTC m=+2696.678344875"
Jul 18 14:17:18.313932 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:18.313898    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:18.314363 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:18.313959    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:18.326233 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:18.326211    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:19.051173 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:19.051140    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:17:44.921649 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:44.921614    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"]
Jul 18 14:17:45.901091 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:45.901055    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"]
Jul 18 14:17:56.580489 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.580454    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"]
Jul 18 14:17:56.583578 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.583557    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.585816 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.585797    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1bdd6b\"/\"rate-test-model-1bdd6b-kserve-self-signed-certs\""
Jul 18 14:17:56.594242 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.594217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"]
Jul 18 14:17:56.596692 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.596649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-home\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.596805 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.596697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00911e01-0868-4810-801b-700210d59cb9-tls-certs\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.596805 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.596737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-model-cache\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.596805 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.596764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl5fd\" (UniqueName: \"kubernetes.io/projected/00911e01-0868-4810-801b-700210d59cb9-kube-api-access-cl5fd\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.596953 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.596919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-dshm\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.597024 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.597007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-tmp-dir\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.697750 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.697716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-model-cache\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.697935 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.697759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl5fd\" (UniqueName: \"kubernetes.io/projected/00911e01-0868-4810-801b-700210d59cb9-kube-api-access-cl5fd\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.697935 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.697821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-dshm\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.697935 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.697874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-tmp-dir\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.697935 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.697893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-home\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.697935 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.697909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00911e01-0868-4810-801b-700210d59cb9-tls-certs\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.698212 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.698193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-model-cache\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.698259 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.698232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-tmp-dir\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.698335 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.698312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-home\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.700216 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.700187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-dshm\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.700505 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.700484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00911e01-0868-4810-801b-700210d59cb9-tls-certs\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.704936 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.704919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl5fd\" (UniqueName: \"kubernetes.io/projected/00911e01-0868-4810-801b-700210d59cb9-kube-api-access-cl5fd\") pod \"rate-test-model-1bdd6b-kserve-58f956764c-q4spg\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") " pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:56.893266 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:56.893168    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:17:57.022976 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:57.022944    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"]
Jul 18 14:17:57.025560 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:17:57.025531    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00911e01_0868_4810_801b_700210d59cb9.slice/crio-f645c0a8f2df7a48e43a52ecc4322bc612dcfcc11af4093448a5c0bf8574bdbc WatchSource:0}: Error finding container f645c0a8f2df7a48e43a52ecc4322bc612dcfcc11af4093448a5c0bf8574bdbc: Status 404 returned error can't find the container with id f645c0a8f2df7a48e43a52ecc4322bc612dcfcc11af4093448a5c0bf8574bdbc
Jul 18 14:17:57.225345 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:57.225310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg" event={"ID":"00911e01-0868-4810-801b-700210d59cb9","Type":"ContainerStarted","Data":"412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108"}
Jul 18 14:17:57.225493 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:57.225351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg" event={"ID":"00911e01-0868-4810-801b-700210d59cb9","Type":"ContainerStarted","Data":"f645c0a8f2df7a48e43a52ecc4322bc612dcfcc11af4093448a5c0bf8574bdbc"}
Jul 18 14:17:57.245972 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:57.245923    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg" podStartSLOduration=1.245908607 podStartE2EDuration="1.245908607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:17:57.244324943 +0000 UTC m=+2744.913774647" watchObservedRunningTime="2026-07-18 14:17:57.245908607 +0000 UTC m=+2744.915358312"
Jul 18 14:17:59.941964 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:17:59.941921    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h" podUID="de4127e7-0692-4ace-814c-7e2c2fbbc553" containerName="main" containerID="cri-o://f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2" gracePeriod=45
Jul 18 14:18:00.190237 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.190212    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:18:00.224359 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224334    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-home\") pod \"de4127e7-0692-4ace-814c-7e2c2fbbc553\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") "
Jul 18 14:18:00.224493 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224372    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxnrh\" (UniqueName: \"kubernetes.io/projected/de4127e7-0692-4ace-814c-7e2c2fbbc553-kube-api-access-mxnrh\") pod \"de4127e7-0692-4ace-814c-7e2c2fbbc553\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") "
Jul 18 14:18:00.224493 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224402    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-tmp-dir\") pod \"de4127e7-0692-4ace-814c-7e2c2fbbc553\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") "
Jul 18 14:18:00.224616 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224518    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-dshm\") pod \"de4127e7-0692-4ace-814c-7e2c2fbbc553\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") "
Jul 18 14:18:00.224616 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224545    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4127e7-0692-4ace-814c-7e2c2fbbc553-tls-certs\") pod \"de4127e7-0692-4ace-814c-7e2c2fbbc553\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") "
Jul 18 14:18:00.224616 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224580    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-model-cache\") pod \"de4127e7-0692-4ace-814c-7e2c2fbbc553\" (UID: \"de4127e7-0692-4ace-814c-7e2c2fbbc553\") "
Jul 18 14:18:00.224804 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224621    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-home" (OuterVolumeSpecName: "home") pod "de4127e7-0692-4ace-814c-7e2c2fbbc553" (UID: "de4127e7-0692-4ace-814c-7e2c2fbbc553"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:00.224804 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224730    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de4127e7-0692-4ace-814c-7e2c2fbbc553" (UID: "de4127e7-0692-4ace-814c-7e2c2fbbc553"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:00.224972 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224945    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:00.225078 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224971    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-model-cache" (OuterVolumeSpecName: "model-cache") pod "de4127e7-0692-4ace-814c-7e2c2fbbc553" (UID: "de4127e7-0692-4ace-814c-7e2c2fbbc553"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:00.225078 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.224975    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:00.226817 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.226792    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de4127e7-0692-4ace-814c-7e2c2fbbc553-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de4127e7-0692-4ace-814c-7e2c2fbbc553" (UID: "de4127e7-0692-4ace-814c-7e2c2fbbc553"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:18:00.226924 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.226813    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de4127e7-0692-4ace-814c-7e2c2fbbc553-kube-api-access-mxnrh" (OuterVolumeSpecName: "kube-api-access-mxnrh") pod "de4127e7-0692-4ace-814c-7e2c2fbbc553" (UID: "de4127e7-0692-4ace-814c-7e2c2fbbc553"). InnerVolumeSpecName "kube-api-access-mxnrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:18:00.226983 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.226944    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-dshm" (OuterVolumeSpecName: "dshm") pod "de4127e7-0692-4ace-814c-7e2c2fbbc553" (UID: "de4127e7-0692-4ace-814c-7e2c2fbbc553"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:00.243366 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.243340    2579 generic.go:356] "Generic (PLEG): container finished" podID="de4127e7-0692-4ace-814c-7e2c2fbbc553" containerID="f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2" exitCode=0
Jul 18 14:18:00.243454 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.243372    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"
Jul 18 14:18:00.243454 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.243413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h" event={"ID":"de4127e7-0692-4ace-814c-7e2c2fbbc553","Type":"ContainerDied","Data":"f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2"}
Jul 18 14:18:00.243531 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.243454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h" event={"ID":"de4127e7-0692-4ace-814c-7e2c2fbbc553","Type":"ContainerDied","Data":"966f2520ddc4419134c7e377d7199bc06f4795cb3583ff0777f53e0d10f7a6fc"}
Jul 18 14:18:00.243531 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.243471    2579 scope.go:117] "RemoveContainer" containerID="f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2"
Jul 18 14:18:00.252204 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.252188    2579 scope.go:117] "RemoveContainer" containerID="f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2"
Jul 18 14:18:00.252438 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:18:00.252422    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2\": container with ID starting with f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2 not found: ID does not exist" containerID="f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2"
Jul 18 14:18:00.252493 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.252444    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2"} err="failed to get container status \"f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2\": rpc error: code = NotFound desc = could not find container \"f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2\": container with ID starting with f3ab7191492b4ab336cc279fef2e37c8e826bcbc910512cc850575c75627efa2 not found: ID does not exist"
Jul 18 14:18:00.263052 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.263031    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"]
Jul 18 14:18:00.265690 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.265657    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/sub-test-model-1bdd6b-kserve-6c48c7b549-q252h"]
Jul 18 14:18:00.326397 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.326364    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxnrh\" (UniqueName: \"kubernetes.io/projected/de4127e7-0692-4ace-814c-7e2c2fbbc553-kube-api-access-mxnrh\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:00.326397 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.326392    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:00.326397 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.326402    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4127e7-0692-4ace-814c-7e2c2fbbc553-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:00.326605 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.326411    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4127e7-0692-4ace-814c-7e2c2fbbc553-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:00.849579 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.849546    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de4127e7-0692-4ace-814c-7e2c2fbbc553" path="/var/lib/kubelet/pods/de4127e7-0692-4ace-814c-7e2c2fbbc553/volumes"
Jul 18 14:18:00.922691 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:00.922636    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb" podUID="57610f77-7dde-47dd-adfc-c396737d0e94" containerName="main" containerID="cri-o://e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727" gracePeriod=45
Jul 18 14:18:01.183291 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.183265    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:18:01.234199 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234172    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c87sw\" (UniqueName: \"kubernetes.io/projected/57610f77-7dde-47dd-adfc-c396737d0e94-kube-api-access-c87sw\") pod \"57610f77-7dde-47dd-adfc-c396737d0e94\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") "
Jul 18 14:18:01.234357 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234256    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57610f77-7dde-47dd-adfc-c396737d0e94-tls-certs\") pod \"57610f77-7dde-47dd-adfc-c396737d0e94\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") "
Jul 18 14:18:01.234357 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234340    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-tmp-dir\") pod \"57610f77-7dde-47dd-adfc-c396737d0e94\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") "
Jul 18 14:18:01.234473 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-home\") pod \"57610f77-7dde-47dd-adfc-c396737d0e94\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") "
Jul 18 14:18:01.234530 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234472    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-dshm\") pod \"57610f77-7dde-47dd-adfc-c396737d0e94\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") "
Jul 18 14:18:01.234530 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234498    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-model-cache\") pod \"57610f77-7dde-47dd-adfc-c396737d0e94\" (UID: \"57610f77-7dde-47dd-adfc-c396737d0e94\") "
Jul 18 14:18:01.234981 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234767    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-home" (OuterVolumeSpecName: "home") pod "57610f77-7dde-47dd-adfc-c396737d0e94" (UID: "57610f77-7dde-47dd-adfc-c396737d0e94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:01.234981 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.234941    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57610f77-7dde-47dd-adfc-c396737d0e94" (UID: "57610f77-7dde-47dd-adfc-c396737d0e94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:01.235150 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.235005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-model-cache" (OuterVolumeSpecName: "model-cache") pod "57610f77-7dde-47dd-adfc-c396737d0e94" (UID: "57610f77-7dde-47dd-adfc-c396737d0e94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:01.236494 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.236462    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57610f77-7dde-47dd-adfc-c396737d0e94-kube-api-access-c87sw" (OuterVolumeSpecName: "kube-api-access-c87sw") pod "57610f77-7dde-47dd-adfc-c396737d0e94" (UID: "57610f77-7dde-47dd-adfc-c396737d0e94"). InnerVolumeSpecName "kube-api-access-c87sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:18:01.236643 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.236623    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57610f77-7dde-47dd-adfc-c396737d0e94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57610f77-7dde-47dd-adfc-c396737d0e94" (UID: "57610f77-7dde-47dd-adfc-c396737d0e94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:18:01.237030 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.237010    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-dshm" (OuterVolumeSpecName: "dshm") pod "57610f77-7dde-47dd-adfc-c396737d0e94" (UID: "57610f77-7dde-47dd-adfc-c396737d0e94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:18:01.250402 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.250376    2579 generic.go:356] "Generic (PLEG): container finished" podID="57610f77-7dde-47dd-adfc-c396737d0e94" containerID="e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727" exitCode=0
Jul 18 14:18:01.250472 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.250415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"
Jul 18 14:18:01.250472 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.250455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb" event={"ID":"57610f77-7dde-47dd-adfc-c396737d0e94","Type":"ContainerDied","Data":"e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727"}
Jul 18 14:18:01.250550 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.250495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb" event={"ID":"57610f77-7dde-47dd-adfc-c396737d0e94","Type":"ContainerDied","Data":"5fa1aa04edf942f9240c792baa23f90b5ee9ffeb39d9957c32527f5e1f20fc4b"}
Jul 18 14:18:01.250550 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.250512    2579 scope.go:117] "RemoveContainer" containerID="e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727"
Jul 18 14:18:01.259048 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.259028    2579 scope.go:117] "RemoveContainer" containerID="e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727"
Jul 18 14:18:01.259285 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:18:01.259268    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727\": container with ID starting with e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727 not found: ID does not exist" containerID="e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727"
Jul 18 14:18:01.259334 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.259292    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727"} err="failed to get container status \"e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727\": rpc error: code = NotFound desc = could not find container \"e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727\": container with ID starting with e9d33427346120dcc303372d38cd58586daa6553c0afa57045fdf55ac8f54727 not found: ID does not exist"
Jul 18 14:18:01.270621 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.270598    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"]
Jul 18 14:18:01.274467 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.274446    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/sub-test-model-66d5d1-kserve-66cd8fd479-czjwb"]
Jul 18 14:18:01.335538 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.335509    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:01.335538 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.335533    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:01.335538 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.335544    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c87sw\" (UniqueName: \"kubernetes.io/projected/57610f77-7dde-47dd-adfc-c396737d0e94-kube-api-access-c87sw\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:01.335757 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.335554    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57610f77-7dde-47dd-adfc-c396737d0e94-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:01.335757 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.335562    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:01.335757 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:01.335569    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57610f77-7dde-47dd-adfc-c396737d0e94-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:18:02.847235 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:02.847206    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57610f77-7dde-47dd-adfc-c396737d0e94" path="/var/lib/kubelet/pods/57610f77-7dde-47dd-adfc-c396737d0e94/volumes"
Jul 18 14:18:06.893323 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:06.893279    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:18:06.893323 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:06.893335    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:18:06.905900 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:06.905872    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:18:07.289426 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:07.289394    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:18:29.990809 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.990778    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"]
Jul 18 14:18:29.991256 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.991130    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4127e7-0692-4ace-814c-7e2c2fbbc553" containerName="main"
Jul 18 14:18:29.991256 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.991140    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4127e7-0692-4ace-814c-7e2c2fbbc553" containerName="main"
Jul 18 14:18:29.991256 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.991161    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57610f77-7dde-47dd-adfc-c396737d0e94" containerName="main"
Jul 18 14:18:29.991256 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.991166    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="57610f77-7dde-47dd-adfc-c396737d0e94" containerName="main"
Jul 18 14:18:29.991256 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.991232    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="57610f77-7dde-47dd-adfc-c396737d0e94" containerName="main"
Jul 18 14:18:29.991256 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.991240    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4127e7-0692-4ace-814c-7e2c2fbbc553" containerName="main"
Jul 18 14:18:29.994273 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.994252    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:29.997357 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.997318    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"openshift-service-ca.crt\""
Jul 18 14:18:29.997496 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.997362    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"rate-test-model-66d5d1-kserve-self-signed-certs\""
Jul 18 14:18:29.997496 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.997318    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"kube-root-ca.crt\""
Jul 18 14:18:29.997496 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:29.997433    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-66d5d1\"/\"default-dockercfg-xnxfp\""
Jul 18 14:18:30.003578 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.003557    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"]
Jul 18 14:18:30.088272 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.088238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkqpp\" (UniqueName: \"kubernetes.io/projected/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-kube-api-access-jkqpp\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.088417 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.088287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tmp-dir\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.088417 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.088308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-home\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.088417 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.088377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-dshm\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.088417 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.088402    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-model-cache\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.088561 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.088423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tls-certs\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.189900 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.189865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkqpp\" (UniqueName: \"kubernetes.io/projected/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-kube-api-access-jkqpp\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190069 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.189914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tmp-dir\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190069 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.189935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-home\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190069 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.189965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-dshm\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190069 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.189982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-model-cache\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190069 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.189998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tls-certs\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190340 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.190283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tmp-dir\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190435 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.190411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-model-cache\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.190494 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.190420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-home\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.192281 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.192263    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-dshm\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.192541 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.192525    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tls-certs\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.197683 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.197642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkqpp\" (UniqueName: \"kubernetes.io/projected/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-kube-api-access-jkqpp\") pod \"rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") " pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.304582 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.304520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:30.436412 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:30.436369    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"]
Jul 18 14:18:31.394727 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:31.394690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq" event={"ID":"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69","Type":"ContainerStarted","Data":"7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4"}
Jul 18 14:18:31.395146 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:31.394743    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq" event={"ID":"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69","Type":"ContainerStarted","Data":"bc44ee4b52d19808e945b3356eb4745b9b495c8e4c807474947e32226ce89be0"}
Jul 18 14:18:31.410621 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:31.410578    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq" podStartSLOduration=2.410563531 podStartE2EDuration="2.410563531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:18:31.409514544 +0000 UTC m=+2779.078964249" watchObservedRunningTime="2026-07-18 14:18:31.410563531 +0000 UTC m=+2779.080013236"
Jul 18 14:18:40.305260 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:40.305223    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:40.305657 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:40.305284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:40.317903 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:40.317878    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:18:40.448899 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:18:40.448871    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:19:18.741399 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:18.741353    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"]
Jul 18 14:19:19.591585 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:19.591551    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"]
Jul 18 14:19:33.762846 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:33.762803    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg" podUID="00911e01-0868-4810-801b-700210d59cb9" containerName="main" containerID="cri-o://412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108" gracePeriod=45
Jul 18 14:19:34.022516 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.022461    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:19:34.148132 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00911e01-0868-4810-801b-700210d59cb9-tls-certs\") pod \"00911e01-0868-4810-801b-700210d59cb9\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") "
Jul 18 14:19:34.148290 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148144    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-tmp-dir\") pod \"00911e01-0868-4810-801b-700210d59cb9\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") "
Jul 18 14:19:34.148290 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148180    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-model-cache\") pod \"00911e01-0868-4810-801b-700210d59cb9\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") "
Jul 18 14:19:34.148290 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148222    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-home\") pod \"00911e01-0868-4810-801b-700210d59cb9\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") "
Jul 18 14:19:34.148290 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148257    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl5fd\" (UniqueName: \"kubernetes.io/projected/00911e01-0868-4810-801b-700210d59cb9-kube-api-access-cl5fd\") pod \"00911e01-0868-4810-801b-700210d59cb9\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") "
Jul 18 14:19:34.148290 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-dshm\") pod \"00911e01-0868-4810-801b-700210d59cb9\" (UID: \"00911e01-0868-4810-801b-700210d59cb9\") "
Jul 18 14:19:34.148528 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148394    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00911e01-0868-4810-801b-700210d59cb9" (UID: "00911e01-0868-4810-801b-700210d59cb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.148528 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148481    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-model-cache" (OuterVolumeSpecName: "model-cache") pod "00911e01-0868-4810-801b-700210d59cb9" (UID: "00911e01-0868-4810-801b-700210d59cb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.148613 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148509    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-home" (OuterVolumeSpecName: "home") pod "00911e01-0868-4810-801b-700210d59cb9" (UID: "00911e01-0868-4810-801b-700210d59cb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.148694 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148619    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.148694 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148639    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.148694 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.148652    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.150201 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.150182    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-dshm" (OuterVolumeSpecName: "dshm") pod "00911e01-0868-4810-801b-700210d59cb9" (UID: "00911e01-0868-4810-801b-700210d59cb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.150265 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.150252    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00911e01-0868-4810-801b-700210d59cb9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00911e01-0868-4810-801b-700210d59cb9" (UID: "00911e01-0868-4810-801b-700210d59cb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:19:34.150381 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.150365    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00911e01-0868-4810-801b-700210d59cb9-kube-api-access-cl5fd" (OuterVolumeSpecName: "kube-api-access-cl5fd") pod "00911e01-0868-4810-801b-700210d59cb9" (UID: "00911e01-0868-4810-801b-700210d59cb9"). InnerVolumeSpecName "kube-api-access-cl5fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:19:34.250025 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.249996    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00911e01-0868-4810-801b-700210d59cb9-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.250025 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.250022    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl5fd\" (UniqueName: \"kubernetes.io/projected/00911e01-0868-4810-801b-700210d59cb9-kube-api-access-cl5fd\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.250189 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.250034    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00911e01-0868-4810-801b-700210d59cb9-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.611790 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.611749    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq" podUID="ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" containerName="main" containerID="cri-o://7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4" gracePeriod=45
Jul 18 14:19:34.689886 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.689847    2579 generic.go:356] "Generic (PLEG): container finished" podID="00911e01-0868-4810-801b-700210d59cb9" containerID="412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108" exitCode=0
Jul 18 14:19:34.690044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.689896    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"
Jul 18 14:19:34.690044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.689930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg" event={"ID":"00911e01-0868-4810-801b-700210d59cb9","Type":"ContainerDied","Data":"412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108"}
Jul 18 14:19:34.690044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.689979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg" event={"ID":"00911e01-0868-4810-801b-700210d59cb9","Type":"ContainerDied","Data":"f645c0a8f2df7a48e43a52ecc4322bc612dcfcc11af4093448a5c0bf8574bdbc"}
Jul 18 14:19:34.690044 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.690000    2579 scope.go:117] "RemoveContainer" containerID="412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108"
Jul 18 14:19:34.709842 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.709815    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"]
Jul 18 14:19:34.716079 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.716058    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1bdd6b/rate-test-model-1bdd6b-kserve-58f956764c-q4spg"]
Jul 18 14:19:34.722288 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.722268    2579 scope.go:117] "RemoveContainer" containerID="412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108"
Jul 18 14:19:34.722559 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:19:34.722539    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108\": container with ID starting with 412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108 not found: ID does not exist" containerID="412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108"
Jul 18 14:19:34.722610 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.722573    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108"} err="failed to get container status \"412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108\": rpc error: code = NotFound desc = could not find container \"412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108\": container with ID starting with 412f8269d7df1228f9e3637e2e2af28b9b5a99ba9fb18125befe739d360ae108 not found: ID does not exist"
Jul 18 14:19:34.847762 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.847730    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00911e01-0868-4810-801b-700210d59cb9" path="/var/lib/kubelet/pods/00911e01-0868-4810-801b-700210d59cb9/volumes"
Jul 18 14:19:34.852661 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.852640    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:19:34.955851 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.955774    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tmp-dir\") pod \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") "
Jul 18 14:19:34.956013 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.955885    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-dshm\") pod \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") "
Jul 18 14:19:34.956013 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.955926    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkqpp\" (UniqueName: \"kubernetes.io/projected/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-kube-api-access-jkqpp\") pod \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") "
Jul 18 14:19:34.956013 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.955968    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-model-cache\") pod \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") "
Jul 18 14:19:34.956174 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956012    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-home\") pod \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") "
Jul 18 14:19:34.956174 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956014    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" (UID: "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.956174 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956040    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tls-certs\") pod \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\" (UID: \"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69\") "
Jul 18 14:19:34.956307 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956185    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-home" (OuterVolumeSpecName: "home") pod "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" (UID: "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.956307 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956251    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" (UID: "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.956415 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956363    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-model-cache\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.956415 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956380    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-home\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.956415 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.956389    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tmp-dir\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:34.957924 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.957903    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-dshm" (OuterVolumeSpecName: "dshm") pod "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" (UID: "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:19:34.958023 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.958010    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" (UID: "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:19:34.958115 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:34.958101    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-kube-api-access-jkqpp" (OuterVolumeSpecName: "kube-api-access-jkqpp") pod "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" (UID: "ed2f51f6-a30e-4bba-b2a6-db55e8d12a69"). InnerVolumeSpecName "kube-api-access-jkqpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:19:35.057451 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.057422    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-dshm\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:35.057451 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.057447    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkqpp\" (UniqueName: \"kubernetes.io/projected/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-kube-api-access-jkqpp\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:35.057451 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.057457    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69-tls-certs\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:35.695686 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.695632    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" containerID="7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4" exitCode=0
Jul 18 14:19:35.695686 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.695684    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"
Jul 18 14:19:35.695938 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.695722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq" event={"ID":"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69","Type":"ContainerDied","Data":"7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4"}
Jul 18 14:19:35.695938 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.695765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq" event={"ID":"ed2f51f6-a30e-4bba-b2a6-db55e8d12a69","Type":"ContainerDied","Data":"bc44ee4b52d19808e945b3356eb4745b9b495c8e4c807474947e32226ce89be0"}
Jul 18 14:19:35.695938 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.695788    2579 scope.go:117] "RemoveContainer" containerID="7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4"
Jul 18 14:19:35.704768 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.704752    2579 scope.go:117] "RemoveContainer" containerID="7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4"
Jul 18 14:19:35.705170 ip-10-0-135-180 kubenswrapper[2579]: E0718 14:19:35.705147    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4\": container with ID starting with 7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4 not found: ID does not exist" containerID="7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4"
Jul 18 14:19:35.705262 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.705174    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4"} err="failed to get container status \"7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4\": rpc error: code = NotFound desc = could not find container \"7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4\": container with ID starting with 7e71391109b77ee6d65c9b7afa7f41d06227b502d12a00e0d9c2a9d2863eeda4 not found: ID does not exist"
Jul 18 14:19:35.716850 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.716826    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"]
Jul 18 14:19:35.719942 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:35.719920    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-66d5d1/rate-test-model-66d5d1-kserve-fbb59b7dc-6gpqq"]
Jul 18 14:19:36.847953 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:36.847908    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" path="/var/lib/kubelet/pods/ed2f51f6-a30e-4bba-b2a6-db55e8d12a69/volumes"
Jul 18 14:19:38.355826 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.355787    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"]
Jul 18 14:19:38.356391 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.356371    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00911e01-0868-4810-801b-700210d59cb9" containerName="main"
Jul 18 14:19:38.356473 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.356394    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="00911e01-0868-4810-801b-700210d59cb9" containerName="main"
Jul 18 14:19:38.356473 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.356420    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" containerName="main"
Jul 18 14:19:38.356473 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.356429    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" containerName="main"
Jul 18 14:19:38.356626 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.356540    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="00911e01-0868-4810-801b-700210d59cb9" containerName="main"
Jul 18 14:19:38.356626 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.356553    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2f51f6-a30e-4bba-b2a6-db55e8d12a69" containerName="main"
Jul 18 14:19:38.360484 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.360462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:38.363045 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.363006    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:19:38.363866 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.363816    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 14:19:38.365408 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.365375    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"]
Jul 18 14:19:38.486278 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.486248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ad397d-17c7-4213-b9f5-3b6f5943304b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj\" (UID: \"a7ad397d-17c7-4213-b9f5-3b6f5943304b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:38.587778 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.587744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ad397d-17c7-4213-b9f5-3b6f5943304b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj\" (UID: \"a7ad397d-17c7-4213-b9f5-3b6f5943304b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:38.588334 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.588314    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ad397d-17c7-4213-b9f5-3b6f5943304b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj\" (UID: \"a7ad397d-17c7-4213-b9f5-3b6f5943304b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:38.673279 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.673216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:38.801290 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.801264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"]
Jul 18 14:19:38.803698 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:19:38.803656    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ad397d_17c7_4213_b9f5_3b6f5943304b.slice/crio-8d8fc6f55bcd50316ecbd52c429f21d0b2f18dc0504200f1d65fa8e973c5bfc4 WatchSource:0}: Error finding container 8d8fc6f55bcd50316ecbd52c429f21d0b2f18dc0504200f1d65fa8e973c5bfc4: Status 404 returned error can't find the container with id 8d8fc6f55bcd50316ecbd52c429f21d0b2f18dc0504200f1d65fa8e973c5bfc4
Jul 18 14:19:38.805534 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:38.805517    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 14:19:39.718731 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:39.718697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj" event={"ID":"a7ad397d-17c7-4213-b9f5-3b6f5943304b","Type":"ContainerStarted","Data":"8d8fc6f55bcd50316ecbd52c429f21d0b2f18dc0504200f1d65fa8e973c5bfc4"}
Jul 18 14:19:42.734640 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:42.734605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj" event={"ID":"a7ad397d-17c7-4213-b9f5-3b6f5943304b","Type":"ContainerStarted","Data":"6d45d93150fd2fadbec490612ab266801ce6b4e3cc9fae683075b0526e83f955"}
Jul 18 14:19:42.749445 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:42.749394    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj" podStartSLOduration=1.362066131 podStartE2EDuration="4.749377046s" totalImagesPullingTime="3.387310915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:19:42.747810997 +0000 UTC m=+2850.417260702" watchObservedRunningTime="2026-07-18 14:19:42.749377046 +0000 UTC m=+2850.418826751"
Jul 18 14:19:43.740709 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:43.740645    2579 generic.go:356] "Generic (PLEG): container finished" podID="a7ad397d-17c7-4213-b9f5-3b6f5943304b" containerID="6d45d93150fd2fadbec490612ab266801ce6b4e3cc9fae683075b0526e83f955" exitCode=0
Jul 18 14:19:43.740709 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:43.740703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj" event={"ID":"a7ad397d-17c7-4213-b9f5-3b6f5943304b","Type":"ContainerDied","Data":"6d45d93150fd2fadbec490612ab266801ce6b4e3cc9fae683075b0526e83f955"}
Jul 18 14:19:44.875774 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:44.875749    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:44.936928 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:44.936898    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ad397d-17c7-4213-b9f5-3b6f5943304b-maas-api-service-ca\") pod \"a7ad397d-17c7-4213-b9f5-3b6f5943304b\" (UID: \"a7ad397d-17c7-4213-b9f5-3b6f5943304b\") "
Jul 18 14:19:44.937226 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:44.937206    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ad397d-17c7-4213-b9f5-3b6f5943304b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7ad397d-17c7-4213-b9f5-3b6f5943304b" (UID: "a7ad397d-17c7-4213-b9f5-3b6f5943304b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:19:45.038448 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:45.038372    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ad397d-17c7-4213-b9f5-3b6f5943304b-maas-api-service-ca\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:19:45.751006 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:45.750976    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"
Jul 18 14:19:45.751006 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:45.750988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj" event={"ID":"a7ad397d-17c7-4213-b9f5-3b6f5943304b","Type":"ContainerDied","Data":"8d8fc6f55bcd50316ecbd52c429f21d0b2f18dc0504200f1d65fa8e973c5bfc4"}
Jul 18 14:19:45.751248 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:45.751032    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d8fc6f55bcd50316ecbd52c429f21d0b2f18dc0504200f1d65fa8e973c5bfc4"
Jul 18 14:19:53.430791 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:53.430757    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"]
Jul 18 14:19:53.434443 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:53.434414    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1bdd6b-lpzmj"]
Jul 18 14:19:54.847396 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:19:54.847365    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ad397d-17c7-4213-b9f5-3b6f5943304b" path="/var/lib/kubelet/pods/a7ad397d-17c7-4213-b9f5-3b6f5943304b/volumes"
Jul 18 14:20:00.388633 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.388595    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"]
Jul 18 14:20:00.389030 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.388979    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ad397d-17c7-4213-b9f5-3b6f5943304b" containerName="revoke-keys"
Jul 18 14:20:00.389030 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.388991    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ad397d-17c7-4213-b9f5-3b6f5943304b" containerName="revoke-keys"
Jul 18 14:20:00.389102 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.389066    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ad397d-17c7-4213-b9f5-3b6f5943304b" containerName="revoke-keys"
Jul 18 14:20:00.392050 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.392034    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:00.394146 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.394128    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 14:20:00.394372 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.394356    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:20:00.398402 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.398379    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"]
Jul 18 14:20:00.466543 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.466513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c81a7d-76e3-4828-b8a2-6974525769ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx\" (UID: \"40c81a7d-76e3-4828-b8a2-6974525769ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:00.567204 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.567175    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c81a7d-76e3-4828-b8a2-6974525769ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx\" (UID: \"40c81a7d-76e3-4828-b8a2-6974525769ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:00.567774 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.567755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c81a7d-76e3-4828-b8a2-6974525769ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx\" (UID: \"40c81a7d-76e3-4828-b8a2-6974525769ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:00.704398 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.704312    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:00.832525 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:00.832491    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"]
Jul 18 14:20:00.836283 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:20:00.836259    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c81a7d_76e3_4828_b8a2_6974525769ee.slice/crio-90502789aa669231b9a06e927d5205d03897743e70289bb6c5ac0cfaff97c9c4 WatchSource:0}: Error finding container 90502789aa669231b9a06e927d5205d03897743e70289bb6c5ac0cfaff97c9c4: Status 404 returned error can't find the container with id 90502789aa669231b9a06e927d5205d03897743e70289bb6c5ac0cfaff97c9c4
Jul 18 14:20:01.826859 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:01.826772    2579 generic.go:356] "Generic (PLEG): container finished" podID="40c81a7d-76e3-4828-b8a2-6974525769ee" containerID="2c814b05e4934c89bb7c8625dfef6a8c2275e29a08e5806a6aa317db7f24e47e" exitCode=0
Jul 18 14:20:01.827213 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:01.826851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx" event={"ID":"40c81a7d-76e3-4828-b8a2-6974525769ee","Type":"ContainerDied","Data":"2c814b05e4934c89bb7c8625dfef6a8c2275e29a08e5806a6aa317db7f24e47e"}
Jul 18 14:20:01.827213 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:01.826929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx" event={"ID":"40c81a7d-76e3-4828-b8a2-6974525769ee","Type":"ContainerStarted","Data":"90502789aa669231b9a06e927d5205d03897743e70289bb6c5ac0cfaff97c9c4"}
Jul 18 14:20:02.959467 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:02.959444    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:03.087910 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:03.087836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c81a7d-76e3-4828-b8a2-6974525769ee-maas-api-service-ca\") pod \"40c81a7d-76e3-4828-b8a2-6974525769ee\" (UID: \"40c81a7d-76e3-4828-b8a2-6974525769ee\") "
Jul 18 14:20:03.088182 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:03.088161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c81a7d-76e3-4828-b8a2-6974525769ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40c81a7d-76e3-4828-b8a2-6974525769ee" (UID: "40c81a7d-76e3-4828-b8a2-6974525769ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:20:03.188783 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:03.188760    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c81a7d-76e3-4828-b8a2-6974525769ee-maas-api-service-ca\") on node \"ip-10-0-135-180.ec2.internal\" DevicePath \"\""
Jul 18 14:20:03.838959 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:03.838929    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"
Jul 18 14:20:03.839117 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:03.838930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx" event={"ID":"40c81a7d-76e3-4828-b8a2-6974525769ee","Type":"ContainerDied","Data":"90502789aa669231b9a06e927d5205d03897743e70289bb6c5ac0cfaff97c9c4"}
Jul 18 14:20:03.839117 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:03.839054    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90502789aa669231b9a06e927d5205d03897743e70289bb6c5ac0cfaff97c9c4"
Jul 18 14:20:15.456449 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:15.456414    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"]
Jul 18 14:20:15.458682 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:15.458642    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-66d5d1-h7cvx"]
Jul 18 14:20:16.846533 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:16.846502    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c81a7d-76e3-4828-b8a2-6974525769ee" path="/var/lib/kubelet/pods/40c81a7d-76e3-4828-b8a2-6974525769ee/volumes"
Jul 18 14:20:22.610843 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:22.610808    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-559fc857d4-bjpr8_1a33ac65-e196-4253-a2ac-da78bca23cd5/authorino/0.log"
Jul 18 14:20:27.745635 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:27.745599    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-p8pzv_0bc15568-b421-4094-8108-ee4c76dbb0e7/manager/0.log"
Jul 18 14:20:28.186812 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:28.186734    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-7fgxq_83517afa-a501-4042-9d07-9a759eb74229/copy-manifests/0.log"
Jul 18 14:20:28.193202 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:28.193177    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-7fgxq_83517afa-a501-4042-9d07-9a759eb74229/manager/0.log"
Jul 18 14:20:30.312032 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:30.311984    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-559fc857d4-bjpr8_1a33ac65-e196-4253-a2ac-da78bca23cd5/authorino/0.log"
Jul 18 14:20:30.858302 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:30.858274    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-68zm6_4bbf24cc-282e-4617-874e-acfc7403f7e6/manager/0.log"
Jul 18 14:20:31.621870 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:31.621841    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d65948f8d-px92c_14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4/kube-auth-proxy/0.log"
Jul 18 14:20:32.503883 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:32.503841    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qbcl4_74e04f62-8c93-4f02-852e-da3386dd5bfe/main/0.log"
Jul 18 14:20:32.822614 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:32.822525    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqmqgt_44afb505-930d-4fe9-bfab-72281bc7191c/main/0.log"
Jul 18 14:20:33.035577 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:33.035537    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-b8lqs_6cd6c315-9cc8-4757-b211-af3f7fab8f81/main/0.log"
Jul 18 14:20:41.284444 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:41.284407    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tk9z7_cb23571a-abdf-4063-9693-d9e77e7a118b/global-pull-secret-syncer/0.log"
Jul 18 14:20:41.362933 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:41.362908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j22b9_9bb4daa8-645b-4953-a367-c120e0d6805c/konnectivity-agent/0.log"
Jul 18 14:20:41.435979 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:41.435952    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-180.ec2.internal_34bcd3343349a9ff2d90b37e5a2ddd1b/haproxy/0.log"
Jul 18 14:20:46.759049 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:46.759019    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-559fc857d4-bjpr8_1a33ac65-e196-4253-a2ac-da78bca23cd5/authorino/0.log"
Jul 18 14:20:47.105820 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:47.105707    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-68zm6_4bbf24cc-282e-4617-874e-acfc7403f7e6/manager/0.log"
Jul 18 14:20:48.656080 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.656004    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/alertmanager/0.log"
Jul 18 14:20:48.680334 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.680311    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/config-reloader/0.log"
Jul 18 14:20:48.703642 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.703619    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/kube-rbac-proxy-web/0.log"
Jul 18 14:20:48.725099 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.725082    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/kube-rbac-proxy/0.log"
Jul 18 14:20:48.744766 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.744747    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/kube-rbac-proxy-metric/0.log"
Jul 18 14:20:48.768458 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.768438    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/prom-label-proxy/0.log"
Jul 18 14:20:48.788490 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.788468    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57997edb-1f31-4c21-921a-adbc8026498b/init-config-reloader/0.log"
Jul 18 14:20:48.836707 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:48.836683    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-4584v_4a2e8610-58a1-4df8-8f77-d207ad218433/cluster-monitoring-operator/0.log"
Jul 18 14:20:49.183136 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.183099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v55jc_b5ecce71-a288-4cfe-87ac-64dc3125281f/node-exporter/0.log"
Jul 18 14:20:49.203476 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.203452    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v55jc_b5ecce71-a288-4cfe-87ac-64dc3125281f/kube-rbac-proxy/0.log"
Jul 18 14:20:49.220487 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.220461    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v55jc_b5ecce71-a288-4cfe-87ac-64dc3125281f/init-textfile/0.log"
Jul 18 14:20:49.330141 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.330047    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/prometheus/0.log"
Jul 18 14:20:49.346485 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.346458    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/config-reloader/0.log"
Jul 18 14:20:49.367915 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.367893    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/thanos-sidecar/0.log"
Jul 18 14:20:49.390165 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.390144    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/kube-rbac-proxy-web/0.log"
Jul 18 14:20:49.409069 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.409050    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/kube-rbac-proxy/0.log"
Jul 18 14:20:49.426965 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.426939    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/kube-rbac-proxy-thanos/0.log"
Jul 18 14:20:49.444945 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.444929    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a14678f-3279-4398-8783-b60c5183bc00/init-config-reloader/0.log"
Jul 18 14:20:49.629386 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.629310    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66f8b56d9f-wktj5_57543084-fc6d-4e8d-868a-295e4b1f605a/thanos-query/0.log"
Jul 18 14:20:49.646749 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.646725    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66f8b56d9f-wktj5_57543084-fc6d-4e8d-868a-295e4b1f605a/kube-rbac-proxy-web/0.log"
Jul 18 14:20:49.668772 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.668743    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66f8b56d9f-wktj5_57543084-fc6d-4e8d-868a-295e4b1f605a/kube-rbac-proxy/0.log"
Jul 18 14:20:49.686029 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.686003    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66f8b56d9f-wktj5_57543084-fc6d-4e8d-868a-295e4b1f605a/prom-label-proxy/0.log"
Jul 18 14:20:49.703876 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.703852    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66f8b56d9f-wktj5_57543084-fc6d-4e8d-868a-295e4b1f605a/kube-rbac-proxy-rules/0.log"
Jul 18 14:20:49.727328 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:49.727299    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66f8b56d9f-wktj5_57543084-fc6d-4e8d-868a-295e4b1f605a/kube-rbac-proxy-metrics/0.log"
Jul 18 14:20:50.975408 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:50.975374    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-8lpzd_9a1d26ef-9167-4ae1-9279-fd20187cf877/networking-console-plugin/0.log"
Jul 18 14:20:51.823134 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.822963    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"]
Jul 18 14:20:51.823397 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.823382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c81a7d-76e3-4828-b8a2-6974525769ee" containerName="revoke-keys"
Jul 18 14:20:51.823443 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.823399    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c81a7d-76e3-4828-b8a2-6974525769ee" containerName="revoke-keys"
Jul 18 14:20:51.823496 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.823487    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c81a7d-76e3-4828-b8a2-6974525769ee" containerName="revoke-keys"
Jul 18 14:20:51.826836 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.826820    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:51.828989 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.828964    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bq9sq\"/\"kube-root-ca.crt\""
Jul 18 14:20:51.829813 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.829794    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bq9sq\"/\"openshift-service-ca.crt\""
Jul 18 14:20:51.829896 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.829805    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bq9sq\"/\"default-dockercfg-r9wfw\""
Jul 18 14:20:51.835472 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.835449    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"]
Jul 18 14:20:51.902213 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.902184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-lib-modules\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:51.902340 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.902218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-podres\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:51.902340 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.902238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9wxv\" (UniqueName: \"kubernetes.io/projected/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-kube-api-access-b9wxv\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:51.902429 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.902356    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-proc\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:51.902429 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:51.902412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-sys\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.002995 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.002966    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-sys\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-lib-modules\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-podres\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9wxv\" (UniqueName: \"kubernetes.io/projected/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-kube-api-access-b9wxv\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-sys\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-lib-modules\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-proc\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-proc\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.003424 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.003208    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-podres\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.009993 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.009970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9wxv\" (UniqueName: \"kubernetes.io/projected/cef0cf43-f2e0-4b7b-a1c2-8803e168785b-kube-api-access-b9wxv\") pod \"perf-node-gather-daemonset-9zr7c\" (UID: \"cef0cf43-f2e0-4b7b-a1c2-8803e168785b\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.137000 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.136922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:52.184010 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.183981    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-rd9tj_5a062f3a-343a-437a-b521-ae76e8795f27/download-server/0.log"
Jul 18 14:20:52.269739 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:52.269657    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"]
Jul 18 14:20:52.273211 ip-10-0-135-180 kubenswrapper[2579]: W0718 14:20:52.273184    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcef0cf43_f2e0_4b7b_a1c2_8803e168785b.slice/crio-f71fc5e0afd1a7df6e44b3f7b237b1d98c2732245baf11c9d4969c591a64ede8 WatchSource:0}: Error finding container f71fc5e0afd1a7df6e44b3f7b237b1d98c2732245baf11c9d4969c591a64ede8: Status 404 returned error can't find the container with id f71fc5e0afd1a7df6e44b3f7b237b1d98c2732245baf11c9d4969c591a64ede8
Jul 18 14:20:53.066095 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.066054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c" event={"ID":"cef0cf43-f2e0-4b7b-a1c2-8803e168785b","Type":"ContainerStarted","Data":"73a4fd80a3d7a1fee9417e0a9a64feef2733001438e60f6ae297e925bb9e3da5"}
Jul 18 14:20:53.066460 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.066116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c" event={"ID":"cef0cf43-f2e0-4b7b-a1c2-8803e168785b","Type":"ContainerStarted","Data":"f71fc5e0afd1a7df6e44b3f7b237b1d98c2732245baf11c9d4969c591a64ede8"}
Jul 18 14:20:53.066460 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.066143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:53.082829 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.082783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c" podStartSLOduration=2.082767901 podStartE2EDuration="2.082767901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:20:53.082318315 +0000 UTC m=+2920.751768021" watchObservedRunningTime="2026-07-18 14:20:53.082767901 +0000 UTC m=+2920.752217614"
Jul 18 14:20:53.838443 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.838415    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fhc8x_a9bc7408-0f67-49c3-811b-44dac65c9016/dns/0.log"
Jul 18 14:20:53.856934 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.856910    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fhc8x_a9bc7408-0f67-49c3-811b-44dac65c9016/kube-rbac-proxy/0.log"
Jul 18 14:20:53.917817 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:53.917793    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6fjz4_acde3bc8-b671-4cd9-9b88-886cb18a7a8a/dns-node-resolver/0.log"
Jul 18 14:20:54.512116 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:54.512088    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f6fff_733d86a8-681c-40e8-94bd-1be31f891067/node-ca/0.log"
Jul 18 14:20:55.554333 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:55.554303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d65948f8d-px92c_14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4/kube-auth-proxy/0.log"
Jul 18 14:20:56.359952 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:56.359925    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4nzcb_c16a06c2-6872-4981-bf66-8d9266d0ff44/serve-healthcheck-canary/0.log"
Jul 18 14:20:57.051964 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:57.051936    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjvqs_7cc4025b-f7ea-4d85-99f3-c33b38439416/kube-rbac-proxy/0.log"
Jul 18 14:20:57.068456 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:57.068429    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjvqs_7cc4025b-f7ea-4d85-99f3-c33b38439416/exporter/0.log"
Jul 18 14:20:57.086618 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:57.086589    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjvqs_7cc4025b-f7ea-4d85-99f3-c33b38439416/extractor/0.log"
Jul 18 14:20:59.080786 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:59.080762    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-9zr7c"
Jul 18 14:20:59.496028 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:59.495997    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-p8pzv_0bc15568-b421-4094-8108-ee4c76dbb0e7/manager/0.log"
Jul 18 14:20:59.735028 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:59.734999    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-7fgxq_83517afa-a501-4042-9d07-9a759eb74229/manager/0.log"
Jul 18 14:20:59.751540 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:20:59.751445    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-7fgxq_83517afa-a501-4042-9d07-9a759eb74229/copy-manifests/0.log"
Jul 18 14:21:01.273294 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:01.273266    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8664c48d9f-bbwc6_15813ff0-5384-4c75-ab1e-79dc307b838c/manager/0.log"
Jul 18 14:21:02.160647 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:02.160613    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d65948f8d-px92c_14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4/kube-auth-proxy/0.log"
Jul 18 14:21:03.552165 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:03.552139    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d65948f8d-px92c_14c91f45-c78d-4eb1-a5b1-1a7241fe8bd4/kube-auth-proxy/0.log"
Jul 18 14:21:06.314262 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:06.314231    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-4lm28_64a0b6bb-bb2f-43cf-ac09-0e2babad1c63/migrator/0.log"
Jul 18 14:21:06.335275 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:06.335246    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-4lm28_64a0b6bb-bb2f-43cf-ac09-0e2babad1c63/graceful-termination/0.log"
Jul 18 14:21:07.704994 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.704962    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/kube-multus-additional-cni-plugins/0.log"
Jul 18 14:21:07.723810 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.723784    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/egress-router-binary-copy/0.log"
Jul 18 14:21:07.741327 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.741304    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/cni-plugins/0.log"
Jul 18 14:21:07.758910 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.758885    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/bond-cni-plugin/0.log"
Jul 18 14:21:07.776437 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.776414    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/routeoverride-cni/0.log"
Jul 18 14:21:07.794924 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.794904    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/whereabouts-cni-bincopy/0.log"
Jul 18 14:21:07.811701 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:07.811662    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vj46_e53520f7-2b98-4eb5-8475-08e69062b055/whereabouts-cni/0.log"
Jul 18 14:21:08.121403 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:08.121365    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mrvt5_90ac29a0-18f4-4eba-a4f5-ac9f2778f396/kube-multus/0.log"
Jul 18 14:21:08.215467 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:08.215436    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t7vxw_bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8/network-metrics-daemon/0.log"
Jul 18 14:21:08.233291 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:08.233264    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t7vxw_bc9a612f-fdf3-4dea-b467-69bf8bc5d1f8/kube-rbac-proxy/0.log"
Jul 18 14:21:09.281550 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.281523    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/ovn-controller/0.log"
Jul 18 14:21:09.295538 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.295511    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/ovn-acl-logging/0.log"
Jul 18 14:21:09.321813 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.321789    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/kube-rbac-proxy-node/0.log"
Jul 18 14:21:09.345278 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.345251    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/kube-rbac-proxy-ovn-metrics/0.log"
Jul 18 14:21:09.360186 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.360165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/northd/0.log"
Jul 18 14:21:09.380106 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.380088    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/nbdb/0.log"
Jul 18 14:21:09.398225 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.398192    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/sbdb/0.log"
Jul 18 14:21:09.482190 ip-10-0-135-180 kubenswrapper[2579]: I0718 14:21:09.482159    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mf_51e5c3ea-3f91-4525-986a-fff42a4a58f2/ovnkube-controller/0.log"